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SUMMARY 


This  guide  is  divided  into  three  parts.  The  first  part  is  introductory 
material  which  scopes  the  effort  and  defines  human  engineering  and  human 
factors  engineering.  The  second  part  is  designed  to  be  used  by  Air  Force 
Program  Element  Managers,  System  Program  Offices,  and  contractor  Program 
Managers.  It  is  intended  to  show  the  current  management  aspects  of  the 
human  engineering  process  utilizing  directives,  specifications, 
regulations,  and  pamphlets.  Human  engineering  (HE)  activities  are  de¬ 
scribed  in  general  terms  of  both  what  should  be  done  and  when  it  should  be 
accomplished.  The  practical  value  of  HE  is  discussed  in  the  manager's 
section.  Various  HE  program  management  relationships  are  suggested  also, 
and  the  procedure  for  including  HE  in  the  total  system  effort  is 
presented. 

Tne  third  section  is  provided  to  assist  both  the  Air  Force  HE  personnel 
and  the  contractor  HE  manager  and  user  personnel.  For  the  HE  managers  or 
users  who  have  had  considerable  experience,  it  may  be  used  for  a  review  or 
checklist  to  be  sure  that  they  are  doing  all  of  the  tasks  that  they 
should.  For  users  who  are  new  to  this  type  of  work,  most  of  what  is  pro¬ 
vided  will  be  useful  to  accomplish  their  required  tasks.  Assistance  is 
provided  in  the  following  areas: 

a)  Human  engineering,  documentation  and  requirements  that  should  ap¬ 
ply  to  the  program. 

b)  Source  data  to  find  out  what  HE  effort  is  needeo. 

c)  Necessary  planning  and  scheduling  to  accomplish  the  program. 

d)  Necessary  coordination  between  HE  and  other  disciplines  and  with 
the  contractor  program  manager  as  well. 

e)  Possible  allocation  of  effort  to  consultants  and/or 
subcontractors . 
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f)  Preparation  of  HE  portion  of  the  request  for  proposal. 

g)  Contractor  proposal  preparation. 

h)  Proposal  evaluation. 

i)  Contractor  task  accomplishment. 

j)  Air  Force  monitoring  of  contractor. 

It  is  intended  that  this  guide  be  of  assistance  to  both  Air  Force  and  in¬ 
dustry  management  to  understand  and  utilize  HE.  It  is  further  intended 
that  the  gu’de  be  of  help  to  the  relatively  inexperienced  Air  Force  or  in¬ 
dustry  person  assigned  responsibility  for  HE  in  the  system  acquisition 
process. 


DoO  contractors,  government  activities  and  other  users  of  this  document 
are  invited  to  submit  comments  and  suggestions  for  improvement  to 
AF.AMp.L/HFD,  Wri qht-Patterson  AFB  OH  45433. 


PREFACE 


This  guide  is  the  result  of  work  conducted  under  Air  Force  Aerospace 
Medical  Research  Laboratory  Contract  No.  F33615-79-C-0520  between  2  April 
1979  and  2  December  1979.  The  AFAMRL  technical  contract  monitor  was  Mr. 
JeaTr^M^---Rii^--and,  within  the  Boeinq  Aerospace  Company,  the  program  has  di¬ 
rected  by  Mr.  W.  J.  Hebenstreit  of  the  Engineering  Technology  Crew  Systems 
and  Simulation  Technology  organization.  The  author  is  indebted  to  their 
guidance  and  contributions  as  well  as  the  help  of  numerous  persons  in  the 
Crew  Systems  and  Simulation  Technology  organization. 
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1.0 


INTRODUCTION 


1.1  Purpose  of  Guide 

The  objective  of  this  guide  is  to  provide  assistance  to  human  engineers 
and  managers  in  the  planning,  scheduling,  and  performance  of  HE  (Human 
Engineering)  in  the  system/equipment  acquisition  process.  There  nas  been 
a  long-standing  need  to  assist  the  relatively  inexperienced  Air  Force  or 
industry  person  assigned  responsibility  for  HE  in  the  multiphase  system 
acquisition  process.  There  has  also  been  a  need  to  help  management  in 
both  Air  Force  and  industry  to  understand  and  utilize  HE  in  the  system  ac¬ 
quisition  process. 

Occasionally,  the  relatively  inexperienced  person  assigned  responsioi 1 ity 
for  Human  Engineering  starts  an  inappropriate  Human  Engineering  effort 
with  requirements  for  data  wnich  may  never  be  used.  Human  Engineering 
must  be  considered,  along  with  all  other  disciplines,  for  the  contribution 
it  can  make  to  the  system/equipment  acquisition,  with  each  requirement 
justified,  and  all  unnecessary  requirements  tailored  out. 

Within  the  AF,  or  other  services,  there  has  been  no  recent  documentation 
which  completely  describes  all  Human  Engineering  tasks  which  should  take 
place  during  a  major  system  acquisition  program  (Ref.  2,  AFSCP  800-3)*. 
There  has  been  no  common  or  unified  approach  as  to  what  Human  Engineering 
is  or  how  it  relates  to  other  areas  concerning  the  human  or  other  disci¬ 
plines.  It  is  the  purpose  of  this  guide  to  provide  a  better  understanding 
and  appreciation  of  Human  Engineering  to  help  both  managers  and  the  people 
assigned  Human  Engineering  responsibility  in  the  system/equipment  acquisi¬ 
tion  process  in  the  Air  Force  and  in  industry. 


*Ref  2,  AFSCP  800-3  defines  major  program  as  "A  program  so  designated  Dy 
OSD  normally  having  an  estimated  cost  of  $50  million  in  RDT&E  or  $200 
million  in  production." 
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1.2  Scope  of  Guide 

This  document  is  organized  into  three  sections.  This  section,  1.0 
Introduction  is  intended  for  use  with  both  Sections  2.0  and  3.0.  Section 
2.0,  HE  significance  for  Acquisition  Manager,  is  intended  to  be  cried  by 
both  Air  Force  and  contractor  managers  to  show  current  management  aspects 
of  the  HE  process.  Section  2.0  may  be  used  independently  from  3.0; 
however.  Section  3.0  is  dependent  on  data  in  both  1.0  and  2.0.  Section 
3.0,  HE  Application  During  System  Acquisition,  is  intended  to  present  and 
develop  HE  procedures  throughout  the  major  system  acquisition  process  (Ref 
3,  0MB  Circular  A-i09).  Section  3.0  is  intended  for  use  by  Air  Force  or 
industry  personnel  directly  assigned  to  the  HE  function.  These  include 
both  HE  managers  and  analysts.  The  total  guide  is  directly  applicable  to 
HE  alone  rather  than  the  total  field  of  HFE,  Human  Factors  Engineering, 
(see  definitions  in  following  section).  However,  it  is  the  intent  of  the 
guide  to  present  the  relationship  of  HE  to  the  other  HFE  elements: 
Biomedical,  Manpower  and  Personnel  Requirements,  Training,  and  Human  Fac¬ 
tors  Test  and  Evaluation  (T&E).  Although  the  other  HFE  elements  are 
necessary  to  the  successful  accomplishment  of  tne  acquisition  process,  the 
procedures  for  the  accomplishment  of  Manpower  and  Personnel  Requirements 
and  Training  are  not  included  in  this  guide.  The  relationship  of  HE  to 
other  HFE  elements  and  to  other  disciplines  or  technologies  such  as 
Maintainability,  Safety,  Reliability,  and  Survivability,  are  indicated  in 
all  three  sections  of  the  guide. 

1.3  Human  Engineering  and  Human  Factors  Engineering 

MIL -STD-7 2 1 B  (Ref.  61)  defires  Human  Engineering  as  "The  area  of  human 
facte.,  which  -oplies  scientific  knowledge  to  the  design  of  items  to 
achieve  effective  man-machine  integration  and  utilization."  In  the  Air 
Force.  Human  Engineering  is  defined  in  Air  Force  Regulation  (AFR)  800-15 
as  "the  application  of  knowledge  about  human  capabilities  and  limitations 
to  the  system  or  equipment  design,  to  achieve  desired  system  performance 
requirements  through  the  most  effective  use  of  man's  performance 
capabi 1 ity. 
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Human  Engineering  is  one  of  five  elements  in  the  Human  Factors  Engineering 
and  Management  area  of  system  acquisition.  The  other  elements  are: 
Biomedical;  Manpower  and  Personnel  Requirements;  Training;  and  the  Human 
factors  Test  and  Evaluation  Element. 

Human  Factors  Engineering  in  the  Air  Force  is  a  management  concept  to  en¬ 
sure  the  incorporation  of  its  five  elements  into  the  mainstream  engineer¬ 
ing  and  program  management  effort  of  all  acquisition  programs  and  concep¬ 
tual  studies.  As  such,  Human  Factors  Engineering  is  a  much  broader  term 
than  Human  Engineering.  However,  whenever  the  term  Human  Factors  Engi¬ 
neering  is  used,  it  includes  Human  Engineering.  Again,  Human  Engineering 
is  concerned  with  the  design  and  development  of  the  system  or  equipment 
for  the  best  utilization  of  human  capabilities  and  limitations  in  the 
operation,  control,  maintenance,  or  support  of  the  system.  "The  Biomedi¬ 
cal  Element  includes  every  area  that  requires  provisions  for  the  promotion 
of  health  and  safety  --  and  for  the  protection,  sustenance,  escape,  survi¬ 
val  and  recovery  of  personnel  employed  within  the  total  system 
environment."* 

Tnere  is  some  overlap  with  Human  Engineering  in  the  design  area.  The  Man¬ 
power  and  Personnel  Requirements  "element  includes  the  development  cf  man¬ 
power  and  personnel  requirements  to  insure  that  enough  trained  people  are 
available  to  operate,  maintain,  control,  and  support  the  system  or  equip¬ 
ment".  The  Training  "element  includes  all  training  provided,  conducted, 
or  managed  by  the  using  command,  ATC,  or  the  contractor.  !t  incorporates, 
as  a  minimum,  the  trained  personnel  requirements,  training  plan,  training 
equipment  development,  training,  training  support  data,  and  training 
facilities."  This  element  is  based  on  the  output  of  tne  Manpower  and  Per¬ 
sonnel  Requirements  element.  The  Human  Factors  Test  and  Evaluation  Ele¬ 
ment  "is  part  of  the  system  test  effort  and  will  oe  conducted  as  directed 
in  AFR  80-14.  It  is  concerned  with  determining  whether  Air  Force 
personnel,  with  system  training,  can  in  fact  operate,  maintain,  and 
support  the  system  in  its  intended  operational  environment." 


*A1 1  quotes  in  this  section  are  from  AFR  800-15. 
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2.0  HE  SIGNIFICANCE  FOR  ACQUISITION  MANAGERS 

This  section  is  prepared  for  Air  Force  Program  Element  Manaaers  (Ref.  2, 
AFSCP  800-3),  System  Program  Office  Managers  (Ref  AFSCP  800-3),  and  con¬ 
tractor  Program  Managers.  It  should  be  of  use  to  them  as  a  guide  for  what 
they  need  to  understand  about  HE.  Another  major  section  of  this  guide  de¬ 
scribes  the  details  associated  with  the  several  HE  activities. 

Data  summarizing  HE  requirements  as  contained  in  applicable  directives, 
regulations,  and  specifications  are  included  in  this  section.  HE  activi¬ 
ties  are  described  in  general  terms  of  both  what  should  be  done  and  when 
it  shou'ui  be  accomplished.  The  practical  value  of  HE  is  discussed.  Vari¬ 
ous  HE  program  management  relationships  are  suggested  and  the  procedure 
for  including  HE  in  the  total  system  effort  is  presented. 

2.1  Documented  Requirements 

The  specification  of  HE  requirements  is  critical  to  the  successful  accom¬ 
plishment  of  any  major  program  effort.  These  requirements  are  both  of  a 
directed  and  practical  nature.  Paragraph  2.3  (HE  Value)  presents  many  of 
the  practical  HE  requirements  along  with  their  value.  This  paragraph  pre¬ 
sents  the  documented  HE  requirements  including  their  origins.  These  re¬ 
quirements  are  presented  both  for  the  Air  Force  and  the  contractor; 
however,  the  particular  requirements  which  direct  the  Air  Force  are  more 
general  and  slightly  different  from  the  more  detailed  contractor 
requirements.  The  contractor's  requirements  are  derived  from  Air  Force 
requirements. 

2.1.1  Documented  Air  Fn~ce  Requirements 

These  requirements  (see  Table  2.1-1)  derive  from  Department  of  Defense 
Directive  5000.1,  Subject:  Major  System  Acquisitions  (Ref.  8).  This 
directive  states  that  "the  number  and  skill  levels  of  personnel  required 
and  human  engineering  factors  shall  be  included  as  constraints  in  system 
design.  The  integration  of  the  human  element  and  system  shall  start  with 
initial  concept  studies  and  be  refined  as  the  system  program  progresses  to 
form  the  basis  for  personnel  selection  and  training,  training  devices, 
simulators  and  planning  related  to  human  factors". 
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X  Primary  applicability _ •  Secondary  applicability 


In  accordance  with  AFR  800-3,  Engineering  for  Defense  Systems,  HE  Is  In¬ 
cluded  as  a  significant  part  of  the  program  engineering  tasks.  HE  as  an 
element  of  HFE  is  required  as  a  part  of  the  engineering  effort  throughout 
the  system  life  cycle.  It  uses  data  from,  and  contributes  to,  the  system 
engineering  process  in  developing  specification  requirements. 

AF  Regulation  800-15,  Human  Factors  Engineering  and  Management,  establish¬ 
es  the  total  system  HE  effort.  It  is  applicable  throughout  the  system 
life  cycle.  AFR  800-15,  Paragraph  2  on  policy  states  "HFE  must  be  an 
integral  part  of  the  R&D  planning,  conceptual  study  efforts,  exploratory, 
advanced,  and  engineering  development  projects,  equipment  procurements, 
modifications,  and  system  acquisition  programs  where  the  intended  end 
product  has  human  performance  as  a  integral  part". 

Attachment  1  to  AFR  800-15  (including  AFSC  Supplement  1)  further  indicates 
that  AFSC  snail  establish  a  command  office  of  primary  responsibility  (OPR; 
Ref.  2,  AFSCP  800-3)  for  HFE  and  require  the  proper  subordinate  echelons 
to  designate  their  OPR  for  HFE.  The  PO's  (program  offices;  Ref.  AFSC 
800-3)  will  insure  that  appropriate  HFE  effort  is  planned  for  and 
implemented  in  all  systems  and  equipment  programs  within  the  resources 
allotted  to  the  program.  A  part-  or  full-time  HFE  manager  will  be 
assigned  upon  formulation  of  the  program  office  cadre. 

The  AFSC  product  divisions,  Space  Division  (SD),  Ballistic  Missile  Office 
( BMO) ,  Electronic  Systems  Division  (ESD),  Aeronautical  Systems  Division 
(ASD),  and  the  Armament  Division  (AD)  will  assign  trained  HFE  managers  to 
manage  and  conduct  the  HFE  effort  on  systems  or  equipment  with  substantial 
or  critical  man-machine  interface  elements.  Military  Specification 
MIL-H-46855,  Human  Engineering  Requirements  for  Military  Systems,  Equip¬ 
ment  and  Facilities  (Ref  4)  establishes  and  defines  the  requirements  for 
applying  human  engineering  to  the  development  and  acquisition  of  military 
systems,  equipment  and  facilities. 
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These  requirements  include  the  work  to  be  accomplished  by  the  contractor, 
or  subcontractors  in  conducting  a  human  engineering  effort  Integrated  with 
the  total  system  engineering  and  development  effort.  It  is  not  intended 
that  all  the  requirements  In  MIL-H-46855  should  be  applied  to  every  pro¬ 
gram  or  program  phase.  It  must  be  applied  judiciously  and  tailored  to  fit 
the  program  or  program  phase  and  the  acquisition  strategy  to  achieve  cost 
effective  acquisition  and  life  cycle  ownership  of  defense  material. 

The  associated  data  requirements  are  found  in  DoD  5000. 19-L,  Acquisition 
Management  Systems  and  Oata  Requirements  Control  List,  Data  Item  Descrip¬ 
tions  (DIDs,  Form  1665),  DI-H-7051  through  DI-H-7059  (Ref.  62).  These 
data  items  should  also  be  tailored  and  justified  based  on  the  phase  of 
system  acquisition  and  the  acquisition  strategy  as  approved  by  the  system 
program  manager. 

Military  Standard  MIL-STD-1472  Human  Engineering  Design  Criteria  for 
Military  Systems,  Equipment  and  Facilities  (Ref  9)  is  a  set  of  human  engi¬ 
neering  design  criteria,  principles  and  practices  to  achieve  mission  suc¬ 
cess  through  integration  of  the  human  into  the  system,  subsystem,  equip¬ 
ment,  and  facility,  and  achieve  effectiveness,  simplicity,  efficiency, 
reliability,  and  safety  of  system  operation,  training,  and  maintenance. 

The  specification,  the  data  items,  and  the  standards  are  Tri-Service  and 
Industry  coordinated  and  approved  by  DoD.  The  appendix  to  MIl-H-46355  is 
a  guide  for  tailoring  the  specification. 

2.1.2  Documented  Contractor  Requirements 

Contractor  requirements  are  provided  directly  by  the  contract  statement  of 
work.  Generally,  MIL-H-46855  and  MIL-STD-1472  are  specified  contrar' .al 
documents,  to  which  the  contractor  must  adhere.  The  contract  data  re¬ 
quirements  list  (CDRL:  DD  Form  1423)  would  contain  any  data  items  associ¬ 
ated  with  MIL-H-46855  and  for  wnich  the  Air  Force  wanted  data.  CDRL 
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items  typically  include  the  HE  Program  Plan,  test  plan,  system  analysis 
report,  and/or  progress  report.  In  addition  to  the  documented  require¬ 
ments,  the  contractor  should  be  motivated  to  capitalize  on  Human  Engineer¬ 
ing  to  help  design  and  develop  the  most  efficient,  effective,  and  safe 
system  possible  within  the  cost  and  schedule  imposed, 

2.2  Human  Engineering  Support  in  System  Acquisition 

The  Human  Engineering  effort  includes  participation  in  three  primary  areas 
of  system  development:  analysis;  design  and  development;  and  test  and 
evaluation,  (Ref  4,  MIL-H-46855) .  As  a  part  of  the  design  and  development 
area,  technical  data  procedures  are  often  developed.  All  of  these  areas 
or  activities  are  performed  in  combination  with  considerable  inter-  and 
intra-coordination.  The  coordination  includes  planning  and  scheduling  of 
these  basic  efforts  to  insure  that  the  proper  source  data  are  available  to 
do  the  necessary  work,  the  proper  work  is  performed  at  the  proper  time, 
and  that  the  results  of  the  work  are  provided  to  the  proper  persons.  Fre¬ 
quently,  as  a  result  of  the  work  performed,  an  interactive  effort  is  made 
to  refine  the  Human  Engineering  design  requirements.  For  example,  as  a 
result  of  test  a.id  evaluation,  more  analysis  and  eventual  redesign  may  be 
necessary.  Typical  interaction  relationships  between  Human  Engineering 
areas  and  other  technology  areas  of  system  development  are  shown  in  Table 
2.2-1. 


As  indicated  in  Paragraph  1.3,  Human  Engineering  is  one  of  five  elements 
of  Human  Factors  Engineering.  Figure  2.2-1  illustrates  this  relationship. 

2.2.1  Analysis  Area 

HE  areas  of  work  are  like  other  technology  areas  or  activities  in  that 
there  are  problems  brought  about  by  the  new  system  acquisition  and  these 
problems  are  frequently  solved  by  the  analysis  process  of  breaking  them 
down  into  smaller  and  smaller  elements  to  the  point  where  they  can  be 
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Figure  2.2- 1  Human  Engineering  Relation  To 
Human  Factors  Engineering 
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Tab/*  2.2- 1.  Human  Engineering  Relationship  to  Other  Technologies- 
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handled.  At  the  smaller  element  level,  significant  aspects  of  the  total 
problem  can  be  examined  in  detail.  Answers  to  several  detailed  questions/ 
problems  are  more  easily  obtained  than  answers  to  a  few  top  level  ques- 
t  ons/problems. 

Generally,  the  analysis  process  starts  with  the  system  mission  as  described 
by  a  baseline  scenario.  The  mission  objective  and  functions  that  must  be 
performed  by  the  system  are  identified,  described,  and  sequenced.  These 
functions  are  then  analyzed  to  determine  their  proper  allocation  to  person¬ 
nel,  software,  or  equipment.  Once  allocated,  the  personnel  functions  are 
further  analyzed  to  determine  the  specific  operator/maintainer  tasks  which 
must  be  performed  to  accomplish  the  functions.  The  tasks  are  further  de¬ 
tailed  to  show  estimated  time  and  space  relationships.  Frequently,  person¬ 
nel  performance  reliability  estimates  are  also  provided.  These  analyses 
are  performed  by  the  use  of  several  (Ref.  Para.  3.9.4)  manual  (paper  and 
pencil)  and  automatic  (computer /software)  techniques. 

The  results  of  these  analyses  are  specific  hardware  design  criteria.  When 
applied,  these  design  criteria  will  insure  hardware  compatibility  with 
human  performance  capabilities  and  limitations.  For  example,  human  perfor¬ 
mance  reliability  data  are  used  by  System  Safety  to  fully  develop  the  sys¬ 
tem  safety  fault  trees.  Technical  publications  may  be  initiated  based  on 
the  task  analysis  procedures  data.  Personnel  manning  and  skill  level  docu¬ 
mentation  may  be  established  based  on  the  analyses  data.  Training  data  and 
equipment  may  be  initiated  from  the  analysis  effort.  Table  2.2-1  shows  the 
several  technologies  from  which  HE  analysis  receive  inputs  or  to  which  ap¬ 
plications  data  are  provided.  In  addition  to  those  already  indicated.  Sys¬ 
tem  Engineering  and  Operations  Analysis  frequently  provide  data  from  which 
the  s  i  effort  may  be  initiated.  Crew  Station  Design  receives  the  results 
of  HE  workload  analysis  in  order  to  determine  proper  flight  or  mission  crew 
size.  Figure  2.2-2  illustrates  the  time  period  during  a  major  system  ac¬ 
quisition  in  which  the  analysis  and  other  areas  of  system  development 
efforts  mey  occur  most  usefully. 
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Figure  2.22.  Applicable  Program  Stages  for  Human  Engineering  Areas  of  System  DeveU 


2.2.2  Design  and  Development  Area 

The  purpose  of  this  area  of  work  is  to  provide  the  system  man-machine 
design  which  incorporates  all  necessary  human  engineering  design 
criteria.  If  the  man-machine  interface  design  activity  is  not  provided 
directly  by  HE,  then  it  is  the  job  of  HE  to  supply  appropriate  design  data 
to  the  project  design  organization.  The  required  HE  sign-off  on  drawings 
indicates  drawing  compliance  with  appropriate  HE  design  criteria.  The 
man-machine  interface  design  is  not  limited  to  portions  of  system 
equipment,  but  includes  software  design,  procedures,  work  environments, 
and  facilities  associated  with  the  system  functions  requiring  personnel 
interaction. 

This  area  of  work  is  accomplished  by  converting  the  results  of  the  analy¬ 
sis  activity  into  HE  and  Biomedical  design  criteria.  This  work  is  heavily 
dependent  on  the  selection  of  applicable  MIL-STD-1472  (Human  Engineering 
Oesign  Criteria  for  Military  Systems,  Equipment  and  Facilities)  design 
criteria.  Several  HE  techniques  and  tools  are  used.  These  Include  the 
use  of  drawings,  checklists,  vision  plots,  reach  envelopes,  mockups, 
specifications,  and  various  computer  workstation  modeling  programs.  The 
final  developed  design  is  a  man-machine  interface  that  will  operate  within 
human  performance  capabilities,  meet  system  functional  requirements,  and 
accomplish  mission  objectives. 

There  are  several  disciplines  that  HE  interfaces  with  during  the  detailed 
design  effort  (see  Table  2.2-1).  System  Engineering  and  maintainability 
are  two  of  the  most  important  of  these.  In  fact.  Human  Engineering  should 
be  a  part  of  system  engineering.  Most  maintainability  design  criteria 
are,  in  fact.  Human  Engineering  design  criteria.  The  most  appropriate 
time  during  a  major  system  acquisition  program  In  which  the  HE  design  ef¬ 
fort  may  usefully  occur  is  shown  in  Figure  2.2-2. 
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2.2.3  Test  and  Evaluation  Area 


The  HE  test  and  evaluation  {T&E)  effort  is  important  to  verify  that  the 
man-machine  interface  portion  is  properly  designed  so  that  the  system  can 
be  operated,  maintained,  supported  and  controlled  by  user  personnel  in  Its 
intended  operational  environment.  HE  personnel  must  work  closely  with 
operational,  maintenance,  system  engineering,  logistics,  and  training  per¬ 
sonnel  during  operational  T&E.  HE  T&E  also  provides  HE  performance  data 
and  design  criteria  for  use  in  the  development  of  later,  follow  on  system 
acquisitions  or  modifications. 

There  are  approximately  20  well  known  tools  and  techniques  used  to  perform 
HE  T&E.  These  include  test  observation,  checklists,  worksheets,  environ¬ 
mental  measurement,  system  records  review,  interviews,  questionnaires, 
sound  and  video  tapes,  photography,  event  recording,  physiological 
measure-  ment,  simulation,  and  statistics.  Figure  2.2-2  illustrates  the 
proper  time  in  which  the  HE  T&E  effort  may  usefully  occur  during  a  major 
system  acquisition. 


2.3  HE  Value 

There  are  two  ways  to  prove  the  value  of  a  sound  HE  effort.  One  is  to 
show  positive  results  of  HE  activities,  and  the  other  is  to  show  the  nega¬ 
tive  results  from  the  lack  of  HE.  The  following  material  examines  the 
values  of  the  HE  effort  from  both  viewpoints. 

2.3.1  Benefit  from  HE 

As  with  most  worthwhile  efforts,  it  takes  an  investment  of  money  and  time 
to  gain  eventual  savings,  increased  performance,  safety,  and  user  satis¬ 
faction.  The  investment  in  HE  is  relatively  small  compared  to  other 
areas.  The  return  on  the  investment  Is  relatively  high.  The  Air  Force 
acquires  a  system  which:  (1)  is  designed  to  permit  operator,  control  and 
maintenance  personnel  to  achieve  required  performance;  (2)  minimizes  skill 
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and  personnel  requirements  and  training  time;  (3)  achieves  required  reli¬ 
ability  of  personnel -equipment  combinations;  and  (4)  emphasizes  safe 
operations,  maintenance,  and  control.  Some  of  these  benefits  can  be  seen 
from  Human  Factors  Tests  and  Evaluation  Reports  (Refs.  66,  67,  68,  as 
typical).  Some  typical  examples  of  problems  found  in  various  tests  by 
Human  Factors  Engineering  T&E  people  as  reported  by  Crites  (Ref.  63)  are: 

"a.  Fastener  problems  -  On  the  F - 1 5  maintenance  was  seriously 

delayed  because  the  door  fasteners  would  freeze  into  the  nut 
plates.  As  many  as  20  in  one  door  would  have  to  be  drilled 
out.  Improper  provisioning  of  tools,  material 
incompatibility,  and  ineffective  procedures  contributed  to 
this  problem. 

b.  It  took  up  to  8  hours  to  remove  one  cotter  pin  from  the 
flaperon  actuator  of  the  F - 1 6.  This  was  an  extreme  case  of 
poor  accessioi lity. 

c.  The  nosewneel  landing  gear  door  would  close  inadvertently  up 
to  45  minutes  after  hydraulic  pressure  was  applied.  This 
created  a  serious  hazard  to  those  working  around  the 
aircraft.  We  prepared  an  on-site  training  video  tape  warning 
of  the  hazard,  and  all  contractor  and  Air  Force  personnel 
were  shown  this  hazard  within  24  hours  of  its  identification. 

d.  The  A-l 0  pilots  had  to  lower  their  heads  nearly  10  inches  in 
order  to  use  the  gunsight  after  high  drag  bomb  delivery. 

e.  Ejection  Seat  Failure  to  Deploy  -  The  pilot  of  an  A-7  re¬ 
ported  that  he  had  pulled  the  ejection  handle  hut  the  seat 
failed  to  fire.  Since  we  had  the  same  ejection  seat  in  the 
F-15,  we  wondered  if  an  incorrect  maintenance  procedure  could 
have  accounted  for  the  failure.  The  ejection  seat  personnel 
did  identify  a  design  deficiency  that  would  allow  a  mainte¬ 
nance  man  to  misrig  the  cable  to  the  initiator.  We  made  a 
video  tape  of  a  seat  being  misrigged  and  sent  copies  to  the 
F-15  System  Program  Office  (SPO),  prime  contractor,  seat 
contractor,  and  Life  Support  SPO.  Design  changes  were 
implemented  to  correct  this  deficiency." 


The  ultimate  test  of  value  is  how  well  the  system  performed  its  mission. 

If  the  human  operator,  maintainer,  or  controller  can  perform  his  job 
efficiently,  effectively,  and  safely,  the  system  has  been  well  human 
engineered.  If  there  are  errors  or  accidents  due  to  the  human  element, 
perhaps  the  system  was  not  well  human  engineered. 

Although  HE  cannot  take  sole  credit,  flying  safety  statistics  have  im¬ 
proved  greatly  within  the  past  20  years.  This  is  because  of  the  concerted 
application  of  HE  principles  to  cockpit  design,  as  well  as  other  areas  of 
aircraft  operations  and  maintenance.  Operator  performance  has  been  shown 
to  improve  to  the  point  of  significantly  affecting  overall  system 
performance.  The  difference  between  a  wel 1 -designed,  versus  a  poorly- 
designed,  console  layout  may  be  an  increase  in  overall  operator  reliabili¬ 
ty  by  an  order  of  magnitude.  The  time  required  to  perform  complex  tasks 
may  easily  be  cut  in  half  by  the  application  of  proven  HE  design  criteria. 

2.3.2  Problems  from  Lack  of  HE 

Until  recently,  it  has  been  difficult  to  obtain  detailed  data  directly 
related  to  problems  resulting  from  the  lack  of  HE.  However,  many  of  the 
problems  found  during  T&E  (see  previous  paragraph)  are  evidence  of  the 
lack  of  a  good  HE  effort  during  the  design  and  development  phase.  Some  of 
the  problems  are  resolvable,  but  it  costs  more  to  do  so  during  this 
phase.  Problems  found  during  the  operational  phase  are  still  more  costly 
to  resolve.  Sometimes  problems  are  identified  only  after  a  crucial  hap¬ 
pening  such  as  a  recent  (1979)  F - 16  accident  after  an  aerial  refueling  as 
reported  by  Griffiths  in  Aviation  Week  and  Space  Technology.  "Also  the 
aerial  refueling  door  was  open  at  the  crash  site.  An  Air  Force  official 
said  closing  the  door  after  refueling  is  the  pilot's  responsibility"  (Ref. 
64).  This  is  not  to  imply  that  the  pilot  was  in  any  way  responsible  for 
the  accident,  but  to  show  that  the  system  was  designed  such  as  to  increase 
the  probability  that  the  pilot  would  make  an  error.  Accident  reports 
showing  pilot  error  as  a  direct  or  contributing  cause  of  an  accident  need 
careful  study  to  determine  if  the  design  increases  the  probability  of 
pilot  errors  and  to  modify  the  design  in  such  cases. 
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A  non-Air  Fo:ce  incident  receiving  national  attention  is  worthy  of  mention 
because  it  shows  the  lack  of  HE;  it  is  so  costly,  and  it  has  affected  so 
many  people.  This  is  the  Three  Mile  Island  Nuclear  Power  Plant  problem. 

It  has  provided  pressure  to  bolster  a  HE  effort  in  the  nuclear  power 
industry.  The  accident  investigation  findings  (Ref.  10,  NURE6-0560)  state 
that  "Human  factors  engineering  has  not  been  sufficiently  emphasized  In 
the  design  and  layout  of  the  control  rooms.  The  location  of  instruments 
and  controls  in  many  power  plants  often  increases  the  likelihood  of  opera¬ 
tor  error,  or,  at  the  least,  impedes  the  operator  in  efficiently  carrying 
out  tne  normal,  abnormal,  and  emergency  actions  required  of  him".  With 
this  disregard  for  HE  principles,  it  was  inevitable  that  the  accident 
should  have  occurred.  It  is,  of  course,  difficult  to  obtain  data  as  to 
the  cost  in  total  dollars,  time,  and  effort  lost  because  of  this  accident, 
but  it  is  not  hard  to  imagine  the  small  percentage  cost  of  a  reasonably 
sound  HE  effort  in  comparison.  The  temptation  is  always  present  to  avoid 
this  small  percentage  cost,  and  to  hope  that  power  plant  design  engineers 
have  sufficient  skill  to  incorporate  all  necessary  HE  design  features. 
However,  proper  knowledge  of  HE  principles  and  criteria  is  too  much  to 
expect  without  HE  training.  Typical  HE  design  criteria  violations  which 
have  occurred  in  power  plant  control  room  design  are  as  follows: 

a.  Instrumentation  and  controls  are  located  beyond  the  operator's 
normal  duty  station  and  visual  envelope;  in  some  cases, 
operators’  backs  are  positioned  towards  the  displays  which  they 
must  monitor. 

b.  Displays  are  located  to  allow  erroneous  readings  due  to 
parallax. 

c.  Displays  and  controls  are  mislabeled  according  to  their 
function. 

d.  Displays  and  controls  are  arbitrarily  located  without  function¬ 
al  grouping. 
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e.  Panel  layouts  for  similar  systems  are  designed  as  mirror  images 
of  each  other,  thereby  violating  HE  principles  of  transfer  of 
training. 

f.  Annunciator  audible  warning  systems  are  misused  to  the  point  of 
serving  more  to  rattle  the  operator  and  overload  his  sensory 
mechanisms  than  to  focus  his  attention  on  the  specific  problem 
at  hand. 

Similar  design  deficiencies  have  been  found  in  Air  Force  Systems  — 
missile  systems,  space  systems,  command  and  control  systems,  aircraft 
systems,  and  support  equipment. 

2.4  HE  Program  Management 

Whereas  there  is  little  doubt  or  interpretation  as  to  what  basic  HE  activ¬ 
ities  must  be  performed  by  the  Air  Force  and  the  contractor  during  a  major 
system  acquisition,  there  is  considerable  latitude  allowable  as  to  the  or¬ 
ganizational  relationships  and  management  of  the  contractor  HE  effort.  A 
survey  of  present  day  practice  indicates  a  variety  of  methods  in  which  the 
HE  organization  can  be  established  within  the  contractor  program  organi¬ 
zation.  (Ref.  11,  Geer  1976).  Whatever  the  organization.  Air  Force  or 
contractor,  the  important  factors  are  the  urgent  need  for  HE  to  be  able  to 
communicate  vertically  to  its  management  and  laterally  to  the  other  tech¬ 
nologies  or  program  groups  which  serve  its  needs  and  which  it,  in  turn, 
serves.  Both  the  Air  Force  and  contractor  HE  programs  should  be  coordi¬ 
nated  with  system  engineering,  maintainability,  system  safety, 
reliability,  survivability/vulnerability,  integrated  logistics  support, 
and  other  HFE  functions  including  biomedical,  personnel,  and  training. 

2.4.1  HE  in  Air  Force  Organization 

In  accordance  with  AFR  800-15,  AFSC  maintains  a  command  office  of  primary 
responsibility  (OPR)  far  HFE  and  requires  subordinate  echelons  to  do  the 
same.  Specifically,  Space  Division  (SO),  Ballistic  Missile  Office  (BMO), 
Aeronautical  Systems  Division  (ASO),  Armanent  Division  (AD),  Arnold 
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Engineering  Development  Center  (AEDC),  Air  Force  Flight  Test  Center 
(AFFTC),  and  Air  Force  Eastern  Test  Range  (AFETR)  have  established  HFE 
focal  points  which  perform  and  coordinate  various  HFE  activities  fcr  the 
various  field  command  programs.  Specific  program  offices  ensure  that  the 
appropriate  HFE  effort  is  planned  for  and  implemented  in  all  systems  with 
a  significant  man-machine  interface.  A  part-  or  full-time  HFE  manager 
should  be  assigned  upon  establishment  of  the  program  office  organization. 

It  is  this  HFE  manager's  jOD  to  f.ianage,  monitor,  and  conduct  the  program 
HFE  effort.  AMD  and  AFHRL  also  provide  HFE  personnel  for  consideration 
and  prompt,  effective  support  to  program  offices,  other  AFSC  field  command 
HFE  staff  officers,  and  R&D  managers. 

2.4.?  HE  in  Contractor  pqaization 

The  HE  function  is  found  in  various  places  in  various  contractor 
organizations.  The  function  is  also  described  by  a  variety  of  organiza¬ 
tional  names.  The  two  basic  areas  in  which  HE  operates  are  in  staff 
support  technology  groups  and  in  program  project  design  groups.  Some  of 
the  names  under  which  HE  operates  are  Crew  Systems,  Erqonomics,  Human 
Factors,  Human  Engineering,  Engineering  Psychology,  Behavioral  Sciences, 
Bioengineering,  Biotechnology  and  Bioastronautics. 

Several  aerospace  contractors  do  not  have  engineering  staff  organizations 
from  which  to  obtain  specialized  technology  skills  such  as  HE.  Their  pro¬ 
ject  organizations,  including  all  project  personnel  exist  within  the 
company  f-nly  for  the  purpose  of  the  project.  They  are  hired  for  that  pro¬ 
ject  alone  and  they  are  laid  off  or  completely  reassigned  to  a  new  organi¬ 
zational  group  when  their  function  for  that  project  is  completed. 
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The  specific  relation  of  HE  to  other  groups  within  a  program  project 
varies  in  accordance  with  the  program  RFP  or  the  desires  of  the  program 
manayer.  The  RFP  may  indicate  the  desired  relationship  for  HE  by  the  or¬ 
ganization  of  the  SOW  or  the  WBS  (Work  Breakdown  Structure,  Ref.  31).  The 
Air  Force  may  Informally  request  the  location  of  HE  within  the  project. 

In  any  case,  HE  is  typically  included  as  a  part  of  System  Engineering, 
Product  Assurance,  Logistics  Engineering,  or  Design  Engineering  (Ref.  5, 
AFR  800-3).  Within  System  Engineering,  it  may  be  subsumed  under  Specialty 
Engineering  or  it  may  report  directly  to  System  Engineering.  HE  is  found 
reporting  directly  to  Project  Engineering  only  on  smaller  programs,  not 
major  system  acqusitions. 


2.5  Initial  Application  of  HE  to  Program 

This  section  briefly  describes  the  method  by  which  HE  is  initially  incor¬ 
porated  into  the  major  system  acquisition  process.  Tne  acquisition  pro¬ 
cess  generally  consists  of  five  phases  (Ref.  12,  AFM  11-1),  the  first  of 
which  is  the  Conceptual  Phase.  The  first  major  task  to  accomplish  during 
this  phase  is  the  preparation  of  the  SON  (Statement  of  Operational  Need) 
by  HQ  USAF  or  a  major  operating  command.  (Ref.  13,  AFR  57-1).  Mission 
analysis  is  performed  by  AFSC  at  the  same  time  to  support  the  SON  prepara¬ 
tion  effort.  The  SON  provides  criteria  for  the  developmental  planning  of 
a  specific  program  and  contains  statements  of  an  operational  need  arising 
from  a  described  Air  Force  Mission.  Since  the  SON  serves  as  a  basis  for 
specific  design  planning,  it  is  desirable  that  HE  contribute  to  its  pre¬ 
paration  through  providing  advice  on  the  kinds  of  HE  requirements  that  can 
reasonably  be  made  at  this  stage  in  development.  After  appropriate  review 
of  the  SON  by  AFSC,  HQ  USAF  prepares  a  PMD  (Program  Management  Directive, 
Ref.  2,  AFSCP  800-3).  The  PMD  is  used  to  direct  and  guide  appropriate  ac¬ 
tion  in  the  Conceptual  Phase.  This  includes  the  actions  to  be  performed 
by  the  commands  to  translate  the  SON  into  a  proposal  for  a  new  program. 
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It  is  at  this  point,  the  preparation  of  the  PMD,  that  the  HQ  USAF  Program 
Element  Manager  must  insure  that  HFE  (including  HE)  requirements  are 
included.  Altnough  the  content  of  the  PMD  is  tailored  to  the  needs  of 
each  individual  program,  any  major  system  acquisition  requires  a  signifi¬ 
cant  man-machine  interface  which,  in  turn,  requires  the  application  of  HFE 
requirements.  In  addition  to  oeing  used  to  state  HFE  requirements,  the 
PMO  may  be  used  to  request  special  HE  studies  or  analysis  and  assign 
responsibility  to  specific  organizations. 

The  PMD  requirement  for  HFE  may  be  worded  as  follows: 

In  compliance  with  AFR  800-15,  AFSC  will  insure  that  the  numan 
component  of  the  system  can  safely  and  effectively  operate, 
maintain,  support,  and  control  the  system  in  its  intended  opera¬ 
tional  environment. 

The  implementing  command,  usually  AFSC,  well  then  include  HE  in  their  Form 
56,  AFSC  Program  Directive  (Ref.  2,  AFSCP  800-3).  It  establishes  the  pro¬ 
gram  priority  and  insures  guidance  and  direction  to  the  AFSC  organiza¬ 
tions'  product  divisions  and  Program  Offices.  The  AFSC  Form  56  may  be 
used  to  call  out  the  need  for  specific  HE  laboratory  (e.g.  AFAMRL  HED) 
support.  In  estaDlishing  the  PO  (Program  Office),  AF5C  must  insure  that 
qualified  personnel  are  assigned  the  particular  task  of  implementing  the 
HE  program  requirements. 

Frequently,  the  PMD  is  modified  oy  program  supplements  at  later  points  in 
the  program  schedule.  If  HE  or  HFE  is  not  included  originally  as  a  pro¬ 
gram  requirement,  it  may  be  included  later. 

One  of  the  most  important  of  many  tasks  that  the  PO  must  accomplish  is  the 
preparation  and  update  of  the  PMP  (Program  Management  Plan).  This  plan 
must  include  the  organization  and  functions  of  the  PO,  in  general,  and  the 
particular  role  of  HFE  within  the  PO.  The  PO  establishes  not  just  the  HrE 
manager  but  the  amount  of  HE  effort  to  be  performed  on  the  program.  The 
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Air  Force  manager  will  tailor  the  effort  to  suit  the  program  needs  and 
budget,  later  In  the  acquisition  cycle,  the  contractor(s)  will  prepare 
their  own  PMP  as  a  part  of  the  proposed  program  development  effort.  The 
contractor  PMP  must  include  HE  and  its  organizational  and  functional  rela¬ 
tionship  to  the  several  related  technologies  such  as  listed  in  Table 
2.2-1.  In  similar  manner  to  the  Air  Force,  the  contractor  program  manager 
must  insure  that  the  HE  management  job  is  assigned  and  funded  to  satisfy 
Air  Force  contractual  requirements. 
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3.0  HE  APPLICATION  DURING  SYSTEM  ACQUISITION 

The  purpose  of  this  major  section  of  the  guide  is  to  assist  both  the  Air 
Force  HE  personnel  and  the  contractor  HE  manager  and  user  personnel.  For 
the  managers  or  users  who  have  had  considerable  experience,  it  may  be  used 
for  a  review  or  checklist  to  be  sure  that  they  are  doing  all  of  the  tasks 
that  they  should,  for  users  who  are  new  to  this  type  of  work,  most  of 
what  is  provided  herein  will  be  useful  to  accomplish  their  several  tasks. 
New  HE  personnel  will  find  that  HE  offers  both  variety  and  a  challenge. 

In  general,  the  workload  is  rigorous.  It  is  the  nature  of  HE  to  offer  a 
seemingly  unending  quantity  of  problems  and  opportunities.  The  HE  job  is 
not  like  that  of  designing  a  landing  gear;  such  tasks  tend  to  have  a  defi¬ 
nite  time  at  which  they  may  be  considered  complete.  For  HE  there  is 
really  no  point  at  which  the  job  is  totally  finished.  It  is  the  task  of 
the  Human  engineering  specialist  to  choose  and  work  the  tasks  which  are 
most  significant  to  the  program  at  any  given  point  in  the  acquisition 
process.  The  following  paragraphs  provide  assistance  in  system  acquisi¬ 
tion  areas  of: 

a.  Human  engineering,  documentation  and  requirements. 

b.  Source  data  to  find  out  what  HE  effort  is  needed. 

c.  Planning  and  scheduling  information. 

a.  Coordination  between  HE  and  other  disciplines  and  with  the  con¬ 
tractor  program  manager. 

e.  Possible  allocation  of  effort  to  consultants  and/or 
subcontractors . 

f.  Preparation  of  HE  portion  of  the  request  for  proposal. 

g.  Contractor  proposal  preparation. 

n.  Proposal  evaluation. 

i.  Contractor  task  accomplishment. 

j.  Air  Force  monitoring  of  contractor. 

The  above  activities  are  depicted  in  typical  sequential  order  in  Figure 
3.0-1.  The  Figure  also  shows  which  activities  are  performed  by  the  Air 
Force  and  the  contractor.  The  first  five  activities  must  be  performed  by 
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Figure  3.0 ■  1.  HE  Activities  During  System  Acquisition 
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the  Air  Force.  They  are  not  actually  required  to  be  performed  by  the  con¬ 
tractor  unti ,  the  "proposal  preparation"  activity.  However,  it  is  recom¬ 
mended  that  these  activities,  as  performed  by  the  contractor,  occur  at  ap¬ 
proximately  the  same  time  that  the  Air  Force  is  performing  them.  One  way 
to  accomplish  this  is  with  the  performance  of  contractor  study  contracts. 

This  guide  also  includes  a  section  (2.0)  which  is  intended  to  assist  the 
Air  Force  Program  Element  Manager,  SPO,  and  contractor  Program  Manager. 

It  is  recommended  that  that  section  be  reviewed  also  for  the  purpose  of 
obtaining  a  different  point  of  view  to  the  major  system  acquisition 
process . 


3.1  Documentation  and  Requirements 

The  specification  and  justification  of  HE  requirements  are  critical  to  the 
successful  accompl isnment  of  the  HE  effort.  The  contractor's  require¬ 
ments,  as  indicated  in  appropriate  documentation,  are  derived  from  the  Air 
Force's  requirements.  The  requirements  which  direct  the  Air  Force  are 
more  general  and  slightly  different  from  the  more  detailed  contractor 
requirements . 

3.1.1  Air  Force  Requirements 

As  indicated  in  Paragraph  2.1.1,  these  requirements  derive  from  Department 
of  Defense  Directive  5000.1,  Subject:  Major  System  Acquisitions  (Ref.  8); 
AFR  800-3,  Engineering  for  Defense  Systems  (Ref.  5);  and  AFR  800-15,  Human 
Factors  Engineering  Management  (Ref.  1).  Although  this  guide  is  directed 
toward  the  HE  task  rather  than  the  complete  HFE  effort,  AFR  800-15  ex¬ 
plains  the  relationship  between  the  two  subjects. 

The  job  of  the  Air  Force  HFE  manager  usually  includes  the  HE  task. 
Occasionally,  there  will  be  separate  Air  Force  managers  who  specialize  in 
training,  training  equipment,  personnel  requirements,  or  biomedical  data. 
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As  discussed  In  paragraph  2.1.1,  MIL-H-46855  is  particularly  important  to 
the  Air  Force  and  potential  contractors.  It  is  used  primarily  by  the  Air 
Force  to  place  HE  requirements  into  the  contract  for  contractor  compli- 
? — •  ance.  Paragraph  3.6.2  describes  how  the  Air  Force  may  go  about  tailoring 
this  specification  for  the  HE  portion  of  the  program  RFP.  j 

In  addition  to  the  above  indicated  documents,  there  are  many  others  which 
;  affect  HE  requirements.  All  documents  which  are  applicable  to  the  HE'ef- 
fort  are  listed  .in  Tqble  3.0-J  along  with  the  aspects  of  HE  which  they  are 
related  to  and  the  nature  of  their  appl icabl ity,  either  primary,  or 
secondary. 

3.1.2  Contractor  Requirements 

Contractor  requirements  are  provided  directly  by  the  contract  statement- 
of-v)ork  (SOW),  contract  iine  items,  and  contract  data  requirements  list 
*  (CDRL).  The  following  Paragraphs  3.6  and  3.7  describe  what  the  Air  Force 
should  include  in  the  SOW  and  CDRL  and  how  the  contractor  should  use  them 
when  responding  to  an  RFP.  SOW  items  which  are  a  part'  of  a  major  system 
acquisition  contract,  MIL-H-46855  and  M I L -STD- 1 472.  are  generally  called 
out  as  contractual  documents  whose  requirements  must  be  adhered  to. 

MIL-H-46855  is  the  primary  source  for  HE  program  requir c-ments .  This  spec¬ 
ification  contains  requirements  for  the  performance  of  HE  analysis,  design 
and  development  criteria,  and  T&E. 

MIL-STD-1472,  Human  Engineering  Oesign  Criteria  for  Military  Systems, 
Equipment  and  Facilities,  describes  proper  hardware  design  criteria  that 
must  be  applied  in  order  to  inherently  enhance  operator/maintainer 
performance.  All  other  documents  shown  in  Table  3.0-1  should  be  consider¬ 
ed  by  the  contractor  as  providing  Information  or  design  guidance  only.  OH 
1-3  Is  the  handbook  most  often  referenced  as  a  design  guide  and  provides 
detailed  data  and  amplification  in  its  fulfillment  of  handbook  objec¬ 
tives.  Paragraph  3.6  describes  the  possible  need  to  tailor  MIL-H-46855  to 
be  used  as  guidance  for  particular  phases  of  the  acquisition  process. 
Contractor  requirements  described  by  the  CDRL  are  also  described  in  Para¬ 
graph  3.6.1. 
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Table  3.0- 1.  Document  Applicability  to  Human  Engineering 
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ai-k  oUU-/  Integrated  Logistics  Support 
AFSCP  80-5  Guide  for  Advanced  Development 
AFSCP  800-3  A  Guide  for  Program  Management 
Human  Engineering  Guide  to  Equipment  Design 
DH1-3  Human  Facton  Engineering 


3.2  Mission  Data  Sources 

New  system  programs  need  a  source  or  sources  of  mission  data  from  which  to 
initiate  the  analysis  and  design  efforts.  These  data  are  In  addition  to 
the  knowledge  (described  in  the  previous  section)  of  which  HE  requirements 
are  derived  from  what  documentation.  Mission  data  are  needed  to  provide 
an  overall  background  of  program  data  from  which  to  develop  new  program 
detailed  requirements.  Initially,  new  program  requirements  are  based  on 
particular  previous  program  study  reports  and  requirements  developed  from 
research  and  exploratory  development  program  phases.  The  following  two 
sections  describe  the  sources  of  data  for  Air  Force  personnel  and  for  con¬ 
tractor  personnel. 

3.2.1  Air  Force  Sources 

There  are  essentially  five  sources  of  data  available  to  Air  Force  person¬ 
nel  assigned  to  a  new  acquisition  program.  These  are: 

a.  Data  from  AF  development  and  planning  organizations  on  studies 
determining  feasibility. 

b.  Research  and  development  ( R&D )  data  developed  by  Air  Force  labs' 
system  paper  studies. 

c.  Data  from  other  previously  developed  but  somewhat  similar 
programs . 

d.  Data  obtained  directly  from  the  potential  Air  Force  user 
commands. 

e.  And  if  all  else  fails,  generation  of  the  necessary  program  sys¬ 
tem  analysis  data  from  scratch. 

During  the  normal  evolution  of  an  Air  Force  system  program,  AF  development 
and  planning  organizations  and  laboratories  will  fund  contractors  to  per¬ 
form  or  develop  (or  both)  paper /software  analysis  studies  of  the  various 
proposed  programs  to  help  determine  feasibility.  This  early  (conceptual) 
program  data  should  be  available  for  review  as  study  reports.  The  reports 
should  contain  direct  reference  to  HE,  HFE,  Crew  Systems,  and/or  the 
man-machine  interface.  If  they  do  not,  they  should  contain  at  least  some 
notion  of  the  system  functional  relationships  with  implications  for  the 
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man-machine  allocation.  A  discussion  of  the  planned  crew  system  or  dis¬ 
plays  and  controls  is  generally  available  in  the  documentation.  Mission 
analysis,  including  scenarios,  flight  profiles,  and  possible  time  lines 
will  contain  direct  implications  for  operator  tasks. 

A  second  useful  source  which  will  contain  considerably  more1  detai 1  but 
will  not  be  as  directly  related  to  any  particular  need  will  be  similar, 
previously  developed  programs.  The  chances  are  good  that  requirements  for 
previous  similar  programs  will  be  much  the  same  in  terms  of:  specifica¬ 
tions  and  standards,  planning  and  scheduling,  coordination,  allocation  of 
effort,  RFP  data,  and  methods  of  contract  monitoring.  HE  test  results 
from  the  operational  T&E  effort  may  also  be  useful.  As  a  word  of  caution, 
it  is  recommended  that  before  previous  program  data  are  utilized,  the  suc¬ 
cess  of  the  HE  portion  of  the  program  be  determined.  Perhaps  the  best  way 
to  find  sufficient  previous  program  data  is  to  seek  out  the  HE  managers  of 
those  programs.  Both  the  details  of  what  was  required  for  that  program 
and  the  success  of  the  man-machine  interface  resulting  from  these  require¬ 
ments  should  be  determined. 

Additional  sources  of  advice  upon  running  an  HE  program  are  the  HE 
manager's  boss  and/or  associates  who  have  had  experience  with  major  pro¬ 
gram  acquisitions. 

Regardless  of  previous  experience  on  similar  programs,  the  HE  manager  must 
contact  the  eventual  program  user  command  to  determine  their  problems, 
needs,  and  recommended  solutions.  Questions  such  as  the  following  or 
those  associated  with  the  DSARC  milestone  checklists  (Ref.  14,  DoDD  5000.2 
and  Ref.  2,  AFSCP  800-3,  Attachment  1)  should  be  asked: 

a)  Why  is  the  new  system/program  needed? 

b)  What  trade-offs  were  (should  be)  considered  in  the  man-machine 
functional  allocation? 

c)  What  does  the  user  command  anticipate  the  most  critical  operator 
tasks  to  be? 
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d)  Is  there  any  particular  human  performance  in  terms  of  time  or 
reliability  that  is  required?  This  will  include  these  factors 
to  be  considered:  Will  human  performance  jeopardize  mission 
performance;  will  system  accuracy  be  degraded;  will  there  be 

•delay  beyond  time  limits;  will  improper  operation  lead  to  system 
failure;  will  excessive  maintenance  downtime  result;  will  there 
be  degradation  below  required  reliability;  will  there  be  damage 
to  equipment;  will  system  security  be  compromised;  will  injury 
to  personnel  occur? 

e)  What  crew  system  problems  does  the  user  command  anticipate 
(e.g.,  manning  levels,  skill  levels,  work  loads,  duty  cycles, 
stress?) 

f)  What,  if  any,  solutions  do  the  user  commands  propose  to  solve 
their  problem? 

Although  each  of  these  questions  should  be  asked,  the  responses  should  not 
be  followed  blindly.  It  is  not  the  user  conmand's  job  to  design  the  new 
system.  The  HE  manager  must  remember  it  is  up  to  the  contractor,  with  the 
SPO's  guidance,  to  design  the  new  system. 

If  for  any  reason  the  previous  attempts  to  obtain  source  data  for  the  HE 
program  are  unsucessful,  the  HE  manager  must  generate  these  data  for 
himself.  He  may,  of  course,  call  upon  Air  Force  lab  support  if  budget  and 
personnel  are  available.  The  HE  manager  can  start  with  the  new  program 
objective  and  the  top  level  functions,  as  described  in  the  SON  (Ref.  Sec¬ 
tion  2.5).  These  functions  have  been  defined  in  crder  for  the  program  to 
have  been  initiated.  From  these  functions,  lower  level  functions  may  be 
generated  along  with  mission  profiles  and  scenarios.  The  development  of 
these  data  is  time  consuming  but  is  very  necessary  in  order  to  proceed 
with  the  program  with  a  knowledge  of  the  significant,  functions  which  af¬ 
fect  the  human  engineering  portion. 
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Part  of  the  Air  Force  HE  manager's  job  is  to  monitor  specific  technology 
areas  continually  for  new  man-machine  interface  concepts,  e.g.,  automated 
speech  technology.  He  should  not  have  to  start  to  develop  or  gather  new 
technology  data  at  the  last  moment.  He  must  also  stay  abreast  of  major 
program  decisions,  made  at  the  higher  levels,  in  order  that  adequate  HE 
research  efforts  can  be  planned  and  implemented. 

3.2.2  Contractor  Sources 

The  source  of  data  from  which  the  contractor  HE  effort  starts  on  a  new 
program  varies  in  accordance  with  the  system  development  phase.  For  the 
conceptual  phase  little,  if  any,  human  engineering  data  will  exist  which 
can  be  used  directly  to  develop  task  analysis  or  man-machine  hardware 
concepts.  It  will  be  necessary  for  the  HE  specialist  to  initiate  develop¬ 
ment  of  these  data  (e.g.,  functional  flow  diagrams)  from  top  level  system 
functions  and  the  mission  objectives.  Paragraph  3.9.4  (analysis)  of  this 
guide  describes  how  tnis  should  be  done.  If  the  HE  effort  Is  addressed  to 
the  advanced  development  phase,  several  alternatives  should  exist  for  the 
contractor  to  obtain  HE  source  data  from  which  to  start  the  effort. 

Source  data  may  be  contained  in  the  RFP  or  included  as  an  attacnment  to 
the  RFP.  Advanced  development  efforts  are  usually  sufficiently  large  that 
several  program  reports  should  be  available  for  gaining  quick  source  data 
information.  The  information  generated  during  the  conceptual  phase  of  the 
program  snould  be  studied  to  determine  the  concepts  for  the  man-machine 
interface.  Many  of  these  reports  are  available  through  the  SPO  while 
others  are  located  at  the  Air  Force  Development  Planning  Organizations  and 
labs  where  the  research  and  exploratory  or  feasibility  work  was  conducted 
or  monitored.  The  contractor  should  not  hesitate  to  ask  the  Air  Force 
customer  for  any  of  his  sources  of  existing  HE  related  program  data.  The 
type  of  general  program  data  which  should  oe  of  assistance  to  HE  users  are 
the: 
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a.  Mission  Objective 

b.  System  Requirements 

c.  Operational  Performance  Requirements 

d.  Environmental  Factors 

e.  Mission  Analysis 

f.  Information  Flow 

g.  Functional  Flow 

If  there  is  a  general  lack  of  program  data  availability  to  a  potential 
contractor  during  a  competitive  program  phase,  and  this  lack  is  relatively 
independent  of  security  classification  considerations,  this  should  be  an 
indication  that  the  potential  contractor  should  not  bid  the  particular 
program  effort.  As  most  contractors  know,  it  is  most  difficult  for  them 
to  initiate  a  major  acquisition  program  without  having  performed  a  signif¬ 
icant  role  in  the  preliminary  research  and  explc ration  phases  of  the  total 
program.  The  time  to  start  gaining  technical  expertise  is  long  before  the 
RFP  is  issued. 

Two  additional  sources  of  HE  data  are  from  previous  similar  programs  and 
from  contractor  HE  personnel  who  have  worked  on  those  programs.  Previous 
similar  program  data  should  be  examined  because  the  methodology  used  to 
provide  the  HE  data  will  probably  be  applicable  to  the  new  program.  Often 
certain  operator  functions  or  tasks  on  a  new  program  are  nearly  identical 
to  those  on  a  previous  program.  HE  managers  or  analysts  should  be  con¬ 
tacted  to  find  out  what  documentation  exists  in  total  for  the  previous 
programs.  They  may  be  able  to  describe  particular  problems  and  solutions 
that  may  not  have  been  documented  but  would  be  most  appropriate  to  the  HE 
effort  on  the  new  program. 

After  contract  award  (assuming  the  contract  is  single  source)  the  contrac¬ 
tor  may  discuss  In  detail  the  availability  of  source  data  with  both  the 
SPO  and,  with  the  SPO's  approval,  the  user  command.  If  not  already 
answered,  questions  such  as  those  in  Paragraph  3.2.1  may  be  presented  in 
order  to  gain  a  better  understanding  of  the  program  HE  problems,  needs  and 
solutions. 


43 


3.3  Planning  and  Scheduling 

Planning  and  scheduling  information  is  just  as  important  to  the  program  as 
the  previously  presented  program  mission  source  data.  Planning  and  sched¬ 
uling  information  is,  however,  considerably  easier  to  obtain  than  the  mis¬ 
sion  source  data.  A  budget  sufficient  for  performing  and  monitoring  the 
HE  effort  is  often  not  as  easily  obtained.  This  paragraph  should  be 
helpful  in  program  planning,  scheduling,  and  budgeting  the  HE  effort. 

3.3.1  Air  Force  Program  Control 

The  program  control  function  will  oe  established  by  the  program  manager 
and  will  include  data  on  planning  and  scheduling  activities  and  on  analy¬ 
sis  of  resources.  This  includes  the  programming  of  contractor,  in-house, 
and  review  activities  so  that  they  mesh  smoothly.  It  also  includes  docu¬ 
mentation  and  management  reporting.  The  major  techniques  used  to  perform 
this  planning  and  scheduling  are  the  event  network  and  the  work  breakdown 
structure  (WBS).  (See  Paragraphs  3. 3. 1.1  and  3. 3. 1.2) 

It  is  the  job  of  the  HE  manager  to  review  the  overall  program  schedule  and 
vJBS  to  insure  that  the  HE  functions  as  described  in  MIL-H-46855  and 
derived  from  his  program  source  data  effort  (Ref.  Paragraph  3.2.1)  will 
occur  at  the  proper  time  to  be  compatible  with  the  other  program 
functions.  Every  program  has  unique  scheduling  requirements. 

Programs  may  be  conducted  in  accordance  with  different  management 
procedures,  For  example,  although  the  intent  of  a  program  advanced  devel¬ 
opment  effort  is  to  provide  eventually  an  operational  system,  the  details 
of  the  significant  program  phases  are  notably  different  from  another  type 
of  system  acquisition  which  also  results  in  an  operational  system.  AFSCP 
80-5  (Ref.  29)  and  AFSCP  800-3  (Ref.  2)  describe  each  of  these  management 
procedures.  AFSCP  80-5  descrioes  the  first  case  where  an  advanced  devel¬ 
opment  program  (Research  and  Development)  evolves  from  research  (6.1 
element)  to  exploratory  development  (6.2  and  on  into  advanced  development 
6.3). 
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If  411  goes  well,  the  program  then  proceeds  to  engineering  development 
(6.4)  and  operational  system  development.  AFSCP  800-3  describes  the 
second  case  where  an  acquisition  system  proceeds  through  the  various  major 
program  phases  following  each  of  the  appropriate  DSARC  milestone  reviews 
and  approvals.  The  program  phases  are  conceptual,  v«|lidatiQnt  full-scale 
development,  and  production.  These  phases  correspond  to  the  last  four  of 
the  above  five  research  and  development  program  phases.  The  difference 
between  these  two  methods  of  program  management/scheduling  is  that  the 
research  and  development  programs  (AFSCP  80-5)  emphasizes  experimental 
-system  demonstrations.  They  involve  the  development  and  test  of  advanced 
systems  of  experimental  design  to  demonstrate  operational  feasibility  or 
increased  operational  capability.  There  are,  of  course,  programs  which 
are  combinations  of  the  two  described  here.  They  may  be  managed/scheduled 
by  either  system  (i.e.,  the  80-series  research  and  development  regulations 
and  pamphlets  or  the  800-series  acquisition  management  procedures  regula¬ 
tions  and  pamphlets). 

It  is  particularly  important  that  the  Air  Force  HE  manager  understand  the 
type  of  program  schedule  so  that  he  may  understand  the  time-phased  need 
for  the  major  portion  of  the  HE  effort.  Advanced  development  programs,  by 
definition,  have  their  major  design  reviews  during  the  6.3  advanced  devel¬ 
opment  phase.  Other  programs,  run  in  accordance  with  the  800-series  regu¬ 
lation  procedures,  have  their  design  reviews  during  a  corresponding  later 
stage,  6.4  full-scale  development.  The  major  HE  effort  must  occur  during 
the  design  review  phase  of  the  program.  However,  it  should  be  noted  that 
it  is  seldom  too  soon  to  initiate  the  HE  portion  of  any  program  and  AFSCP 
800-3  specifically  calls  out  the  need  for  design  reviews  and  HE  planning 
during  the  validation  phase,  prior  to  the  preliminary  and  critical  design 
review  phases  (full-scale  development).  ASDP  800-2  (Ref.  30)  should  be 
useful  to  both  ASO  personnel  and  others  to  understand  HE  planning  and 
integration  into  the  major  acquisition  process.  It  includes  a  flow  chart 
which  depicts  general  milestones  and  indicates  where  ASD  human  factors 
makes  contributions. 
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Figure  3.3  1.  Sample  Work  Breakdown  Structure  (WBS) 


3. 3. 1.1.  Work  Breakdown  Structure 

Th e  WBS  Is  a  numbered  and  indentured  list  of  the  development  efforts  to  be 
conducted  in  the  program,  their  subdivisions,  and  their 
interrelationships. 

The  WBS  is  useful  to  both  the  Air  Force  and  contractors  for  planning, 
costing,  and  scheduling.  The  format  is  determined  by  the  SPO  and 
MIL-STD-881A  (Ref.  31).  It  should  reflect  the  specific  goals  of  the  pro- 
gram  and  the  resources  available  to  meet  them.  Figure  3.3-1  shows  a 
partial  example  of  a  WBS  for  a  hypothetical  program.  The  location  of  HE 
or  HFE  in  relation  to  the  other  WBS  elements  may  vary  considerably  from 
program  to  program. 

3. 3. 1.2  Event  Network 

The  event  network  is  a  time  phased  work  diagram.  It  is  prepared,  based  on 
an  analysis  of  the  WBS  and  an  analysis  of  the  seauence  of  tasks  and 
reviews  required  to  carry  out  the  proposed  development  efforts.  Each 
phase  of  the  program  should  be  broken  down  into  blocks,  each  representing 
a  discrete  event.  A  discrete  event  is  a  portion  of  the  program  involving 
a  single  function,  such  as  review,  test,  design  and  engineering,  or 
procurement,  or  in  some  cases  two  or  more  closely  coordinated  functions, 
performed  by  a  single  group,  such  as  a  contractor  or  an  Air  Force 
facility,  in  a  period  of  time  that  is  also  a  discrete  unit  in  the  total 
sequence  of  events.  That  is,  a  discrete  event  may  take  place  at  the  same 
time  with  other  events  or  in  series  with  the  other  units  chronologically. 
Thus,  similar  functions  may  be  repeated  in  the  various  phases.  In  such 
cases,  they  are  listed  as  separate  events  in  the  network. 

The  event  network  should  identify  the  following  items: 

a)  The  flow  of  events,  including  those  that  are  performed  in 
parallel  and  those  performed  in  sequence  with  other  events. 

b)  The  program  functions  to  be  performed  in-house  Dy  the  Air  Force 
and  those  to  be  performed  by  contractors. 

c)  The  level  (OSD,  HQ  USAF,  HQ  AFSC,  or  other)  of  each  guidance 
and  review  task. 
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3. 3. 1.3  System  Baseline 

In  aadition  to  the  WBS  and  event  network,  another  program  management  tool 
that  must  be  monitored  is  the  system  baseline.  This  is  a  description  of 
the  system  being  developed  in  terms  of  program  requirements*  design 
requirements,  and  configuration.  These  aspects  of  the  baseline  may  be 
established  at  any  point  in  a  development  effort  to  define  a  formal  . 
departure  point  for  control  of  future  changes  in  the  program  or  the  design 
of  hardware.  The  baseline  is  documented  by  approved  program  and  contract 
documents  and  by  specifications  and  drawings  prepared  by  the  contractor. 

3. 3. 1.4  Air  Force  Budget 

The  Air  force  HE  manager* s  budget  is  of  major  concern  in  terms  of  what  he 
can  do  to  monitor  the  program  and  what  support  he  may  obtain  from  Air 
Force  laps.  His  duty  as  system  acquisition  HE  manaqer  is  generally  in  ad¬ 
dition  to  other  duties.  The  problem  of  budget  is  therefore  a  personnel 
one  of  percent  time  allocation  to  a  job  rather  than  total  man-hours 
available.  The  Air  Force  HE  manager  determines  tne  contractor's  budget 
indirectly  in  that  the  more  tasks  he  requires  the  contractor  to  perform  as 
part  of  the  contract,  the  more  budget  the  contractor  HE  personnel  must 
have.  There  is  a  secondary  effect  on  the  Air  Force  HE  manager  in  chat  the 
more  tasks  the  contractor  performs  the  more  Air  Force  review  is  required. 

3.3.2  Contractor  Program  Control 

During  recent  years,  the  scheuuling  and  budget  aspects  of  system 
acquisitions  has  become  paramount,  even  to  the  cost  of  system  performance, 
if  necessary.  In  order  to  maintain  complete  control  of  total  program 
scheduling,  program  subsystem  and  discipline  managers  mu^t  schedule  their 
particular  tasks  in  relation  to  the  major  tasks/events  of  the  total 
program.  Overall  program  control  is  established  by  the  contractor  program 
manager.  This  includes  analysis  and  design  review  activities,  WBS, 
documentation,  and  management  reporting. 
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The  HE  manager  win  perform  HE  planning  and  scheduling  by  starting  with 
the  total  program  milestone  chart.  He  will  add  the  HE  data  requirements 
from  the  CDRL  and  the  HE  tasks  from  MIl-H-46855.  In  general,  these  tasks 
should  Include  operations  analysis,  definition  and  allocation  of  system 
functions,  potential  equipment  selection,  task  analysis,  design  criteria 
application,  equipment  procedures  development,  test  and  evaluation,  and 
any  significant  studies  or  simulations.  Inputs  and  outputs  of  these  tasks 
should  be  included.  The  chart  should  be  started  by  scheduling  HE  products 
at  the  latest  time  that  they  can  be  used  effectively.  The  start  points  and 
time  span  for  HE  analysis  and  other  tasks  by  estimating  the  time  it  will 
take  to  complete  each  task.  If  manpower  utilization  has  not  been  planned, 
an  approximate  estimate  should  be  made  based  on  previous  program  experi¬ 
ence  (yours  or  others).  8ased  on  the  HE  task  start  times,  schedule  all 
data  inputs  to  the  HE  tasks.  This  first  schedule  may  not  work  but  it  is  a 
necessary  starting  point  for  iterations.  Manpower  needs  may  have  to  be 
adjusted;  some  tasks  may  be  reduced  to  meet  the  schedule  requirements  of 
the  overall  program. 

3.3.2. 1  Contractor  8udget 

The  recommendation  of  accurate  manpower  required  to  perform  the  HE  program 
tasks  is  one  of  the  most  needed  and  most  difficult  portions  of  this  guide 
to  provide.  The  best  teacher  of  task  man  loading  is  experience.  The  fol¬ 
lowing  chart.  Figure  3.3-2,  has  been  developed  to  assist  HE  managers  who 
are  new  to  the  job  of  estimating  HE  work  level  effort  in  relation  to  anal¬ 
ysis  and  design  tasks  to  be  performed.  At  best,  the  chart  must  be  consid¬ 
ered  as  not  precise.  There  are  too  many  variables  involved  to  lay  out  an 
accurate  allocation  of  scheduled  HE  manpower.  If  HE  managers  have  had  any 
experience  with  this  kind  of  budgeting  and  scheduling,  they  may  be  better 
off  to  disregard  the  chart  and  rely  on  their  experience.  The  major 
variables  in  the  chart  are  the  types  of  analysis  and  design  to  be 
performed  and  the  program  schedule.  The  manpower  estimates  have  been  made 
as  percentages  of  total  manpower  available  to  do  the  HE  tasks.  The  avail¬ 
able  manpower  could  vary  from  less  than  1  to  20  perions  depending  on  the 
HE  portion  of  the  total  program  and  the  total  program  size. 


Note:  Assume  no  separate  operations  analysis  effort 

assume  small  precontract  effort  Time  (%  of  schedule  shown) 
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Figure  3.3-2,  Hypothetical  Example  of  Program  Manpower  Estimates 
(up  to  time  of  critical  design  review  (CDRI) 


The  numbers  across  the  top  of  the  chart  represent  percent  of  the  schedule 
shown.  Depending  on  the  program,  they  could  represent  weeks  or  days.  The 
two  milestones  are  the  preliminary  design  review  (PDR)  and  the  critical 
design  review  (CDR.  The  PDR  is  often  referred  to  as  the  initial  design 
review.  It  is  the  point  in  the  schedule  whe'e  the  design  specifications 
and  drawings  receive  preliminary  approval  by  the  customer.  The  CDR,  or 
final  design  review,  is  generally  the  time  at  which  the  design  receives 
the  approval  from  the  Air  Force. 

As  indicated,  there  are  variations  in  the  types  of  HFE  analysis  and  design 
required.  Operations  analysis  may  or  may  not  need  to  be  performed,  de¬ 
pending  on  the  program  organization  and  what  work  has  been  performed  prior 
to  this  effort.  Some  programs  will  require  more  analysis  in  some  areas 
and  less  in  others,  for  example,  programs  with  large  operational  crews 
tend  to  require  more  emphasis  on  man-machine  functional  allocations  and 
workload  analysis. 

A  rule  of  thumb  that  is  frequently  used  by  contractors  as  a  budget  start¬ 
ing  point  for  the  HE  effort  is  1%  of  the  total  initial  6.2  exploratory 
development,  if  there  was  one,  or  6.3  advanced  development  fo'-  large 
programs.  There  are  several  variables  that  can  increase  or  decrease  this 
budget  amount.  It  assumes  a  complete  HE  effort  in  accordance  with 
Mll-H-46855  and  MIl-STO-1472.  It  assumes  an  average  size  operational  and 
maintenance  crew.  As  the  program  evolves  into  6.4  full  scale  engineering 
development,  this  percentage  drops  significantly  due  primarily  to  the 
higher  expenditures  for  FSED  rather  than  a  diminished  HE  effort.  The 
single  largest  variable  that  affects  the  budget  is  the  contractor  program 
manager.  If  insufficient  budget  Is  provided  to  perforin  all  of  the  HE 
tasks  required  by  the  SOW,  he  must  be  informed  of  the  consequences  of  the 
Inadequate  budget.  If  he  is  not  convinced  (Reference  Section  2.3,  HE 
Value),  priorities  must  be  established  for  each  of  the  HE  tasks  and  the 
total  level  of  effort  must  be  adjusted  accordingly. 
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3. 3. 2. 2  Contractor  Organization 

The  combination  of  planning,  scheduling,  WBS,  and  budget  implies  an  orga¬ 
nization  of  HE  specialists  to  perform  the  work.  T'it  HC  manager  must 
establish  an  HE  organization  which  reports  (indirectly)  co  the  contractor 
program  manager. 

The  HE  manager  who  is  in  charge  of  the  organization  should  be  thoroughly 
experienced  from  significant  man-machine  efforts  on  previous  major  system 
acquisition  programs.  The  HE  manager  should  be  responsible  for  the 
primary  control,  direction,  supervision  and  management  of  the  technical  HE 
aspects  of  the  program.  He  should  perform  himself,  or  direct  the  accom¬ 
plishment  by  personnel  directly  under  his  supervision,  the  technical  tasks 
of  the  HE  program.  The  HE  manager  should  be  responsible  for  the 
implementation  of  the  following  HE  program  tasks: 

a)  Provide  a  single  point  of  contact  for  HE  related  matters. 

b)  Revise  and  provide  input  to  all  plans  and  contractual  documents 
related  to  HE. 

c)  Maintain  approval  authority  on  all  items  related  to  HE 
contained  in  the  CDRL. 

d)  Coordinate  HE  related  matters  with  contractor  program 
management  and  all  program  elements  and  disciplines. 

e)  Provide  for  investigation  and  reporting  of  all  test  ar.d  evalua¬ 
tion  human  initiated  failures  including  all  incidents  and 
accidents  related  to  HE. 

f)  Participate  in  all  system  requirements  and  design  reviews  to 
assure  that:  all  HE  specified  requirements  are  complied  with; 
HE  schedule  and  CDRL  deliveries  are  compatible;  HE  analysis 
techniques  permit  integration  and  use  in  a  cost-effective 
manner;  and  established  HE  criteria  are  consistent  with  cost, 
performance  and  scheduling  requirements. 

g)  Provide  informal  technical  support  to  program  engineering 
activities. 

h)  Participate  in  program  baseline  configuration  control  activi¬ 
ties  including  the  review  and  approval  of  all  system 
configurations  and  changes  thereto  that  involve  the  human  oper¬ 
ator  and/or  maintainer. 
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3.4  Coordination  ...  _____  ,  v  ....  ...  ..  .  •.  . ,.  ....  _ 

Having  determined  what  HC  tasks! -hr*  required  (Sdurde  Wta)  end  ehart  tfce  ' 
program  schedule  it  (Planning  and  Scheduling);  the  HE1  manager'1  must 
coordinate  the  necessary  HE1 program  tasks  with ^the ‘Air  forde  program !mah- ’ 
agar  and  others.  ‘Of  all  the  disciplines  Involved  In  the  design  and  devel¬ 
opment  of -a  weapon  system,  tSE  requires  the  inost  coordination;  primarily  < 
laterally  to  other  disciplines  but  alsd  vertically  -to  management.  Because 
the  HE  "raison  d'etre",  the  hurtan  element.  Is  a  part  of  most  program 
subsystems,  most  program  disciplines  are  significantly  affected,  and 
therefore,  should  require  considerable  coordination. 

3.4.1  With  Program  Manager 

The  Air  Force  HE  manager  must  tell  the  program  manager  what  HL  can  do  for 
the  program.  Included  In  this  should  be  data  as  to  previous  program 
experiences  (Ref.  Section  3.2,  Mission  Data  Sources)  and  scheduling  data. 
The  Air  Force  HE  manager  should  be  sure  that  the  program  manager  under¬ 
stands  the  need  for  MIL-H-46855  and  MI L-STO-1 472  (if  they  are  required). 

In  particular,  the  program  manager  should  understand  the  need  for  contrac¬ 
tor  HE  sign-off  on  all  drawings  having  an  impact  on  the  man-machine  inter¬ 
face  (Ref.  Section  3.5  of  MIL-H-46855) .  The  knowledge  that  the  Air  Force 
program  manager  will  support  this  requirement  will  assure  that  the  con¬ 
tractor  program  manager  will  adhere  to  it  and  the  resulting  hardware 
designs  will  indeed  include  the  necessary  HE  design  criteria. 

If  there  is  any  problem  with  the  Inclusion  of  HE  and  HFE  items  in  the  WBS, 
they  should  be  discussed  with  Air  Force  program  manager  and  the  Air  Force 
program  control  personnel  as  soon  as  possible.  The  WBS  created  by  the  Air 
Force  should  dictate  that  used  by  the  contractor.  Any  problems  in  the 
original  will  only  cause  problems  for  the  contractor  HE  effort  later  on. 
The  WBS  Indenture  level  that  calls  out  HE  should  be  as  high  as  possible  in 
order  to  provide  emphasis  on  the  importance  of  the  effort. 
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.  5 

;  In  similar  manner  to  the  above,  the  contractor  HE  manager  must  coordinate 

|  with  the  contractor  program  manager  to  Insure  he  has  sufficient  budget. 

i 

;  The  contractor  HE  manager  must  tell  the  program  manager  what  HE  can  do  for 

:  the  program.  Included  In  this  should  be  data  as  to  previous  program 

•  experiences  and  scheduling  data.  The  need  for  Mll-H-46855  and 

i 

j  MIl-STO-1472  (If  they  have  been  called  out)  should  be  explained.  In 

particular,  the  program  manager  should  understand  the  need  for  HE  sign-off 
on  all  drawings  having  an  Impact  on  the  man-machine  interface  (Ref.  3.5  of 
‘  MIL-H-46855) .  This  requirement  must,  of  course,  be  supported  by  the  Air 

Force  program  manager. 

i 

3.4.2  With  Other  Technologies 

The  HE  effort  affects  every  portion  of  the  total  system  that  has  a 
man-machine  interface.  HE  personnel  apply  the  operator/malntainer  capa- 

'  \ 

billties  and  limitations  in  studies  and  specifications  to  the  design  and 
development  of  the  weapon  system  and  its  support  equipment.  Upon 
initiation  of  full-scale  engineering  development,  contractor  HE  organiza- 

{  tions  frequently  assign  specific  HE  personnel  to  support  specific  project 

► 

I  design  organizations  (e.g.,  avionics,  crew  station  design,  or  communica- 

|  tions).  In  this  way  the  individuals  may  become  particularly  expert  at 

dealing  with  particular  types  of  HE  problems  associated  with  particular 
design  groups  (e.g.,  speech  interference  levels  and  communication 
problems).  Appropriate  HE  design  criteria  for  each  type  of  hardware  will 
be  correctly  applied. 

i 

In  coordination  with  personnel  requirements  specialists,  HE  will  use  the 
operator/malntainer  task  analysis  to  develop  manning  requirements  to  oper¬ 
ate  and  maintain  the  weapon  system.  HE  will  participate  in  trade  studies 
to  arrive  at  the  most  efficient  and  cost  effective  man-machine  interface. 
Typically,  HE  will  also  work  with  training  specialists  to  develop  the  re¬ 
quired  skill  and  numbers  of  personnel,  the  training  and  training  support 
necessary  for  the  operation  and  maintenance  of  the  entire  system.  HE 
works  with  his  medical  personnel  on  personnel  safety  and  life  support 
matters.  Coordination  with  such  disciplines  as  system  safety, 
maintainability,  and  reliability  is  not  just  to  ensure  that  the  necessary 
system  requirements  are  met  but  that  they  are  not  duplies :d  by  more  than 
one  group. 
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This  coordination  Is  to  Insure  that  other  disciplines  are  receiving  the 
proper  support  from  HE  and  vice  versa.  In  addition  to  the  program 
manager,  the  disciplines/technologies  illustrated  In  Table  2.2-1  should  be 
contacted  to  inform  them  of  the  analysis,  design,  preliminary  procedures, 
and  test  support  that  HE  has  to  offer.  The  HE  effort  must  be  integrated 
Into  the  total  system  program.  Table  3.4-1  shows  the  relationship  of  sev¬ 
eral  Important  HE  functions  to  other  related  program  functions  and  to  the 
major  acquisition  phases  as  defined  by  three  sources.  The  deployment 
phase,  which  is  defined  by  AFM  11-1  (Ref.  12),  is  not  shown  on  this 
chart.  Both  a  typical  and  important  example  of  such  coordination  would  be 
the  inputs  to  HE  in  regard  to  mission  operations  analysis  or  outputs  from 
HE  analysis  as  to  the  proper  crew  size  for  a  multi-engine  aircraft.  If 
there  are  subsystems  which  will  be  severely  affected  by  the  results  of  the 
HE  effort,  the  appropriate  Air  Force  managers  should  be  forewarned.  It 
is,  cf  course,  up  to  the  Air  Force  HE  manager  to  see  that  the  particular 
HE  analysis,  design,  or  test  effort  is  well  documented  for  presentation  to 
the  affected  subsystem  group. 

3.4.3  With  Other  Services 

Although  not  required,  coordination  with  both  the  Army  and  Navy  manager 
personnel  is  strongly  recommended.  There  is  sufficient  probability  that 
either  they  or  the  Air  Force  would  benefit  by  the  interchange  of  data  on 
similar  aspects  of  their  different  programs.  Both  methodologies  and 
design  requirement  solutions  should  be  discussed.  The  participation  in  HE 
tri-service,  NASA  and  industry  conferences /meetings  is  encouraged  for  the 
exchange  of  useful  data. 

3.5  Allocation  of  Effort 

The  normal  allocation  of  the  HE  effort  is  directly  from  the  Air  Force 
through  the  contract  SOW  to  the  contractor.  However,  there  are  several 
possible  variations  on  this.  The  following  subsections  present  a  few 
alternatives  to  the  normal  HE  work  allocation. 
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Table  3.4 *  /.  Relation  of  HE  Functions  to  Other  Program  Functions  and  Acquisition  Phases 
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3.5.1  Air  Force  Allocation  , 

A1 though  major  system  requirements  are  generally  assigned  for  development 
by  major  contractors,  the  assignment  of  all  HE  functions  to  the  same  con¬ 
tractor  is  not  automatic.  The  major  advantage  in  keeping  the  allocation 
of  HE  tasks  with  the  prime  contractor  Is  simply  minimization  of  the 
coordination  effort.  However,  It  is  possible  that  the  major  contractor 
does  not  have  the  capability  to  perform  a  complete  or  even  a  partial  HE 
effort.  The  contractor  may  propose  the  apportionment  of  HE  tasks  to  other 
sources.  Or  the  Air  Force  HE  manager  may  decide  that  the  best  capability 
to  perform  certain  HE  tasks  exists  within  Air  Force  labs  or  test  centers. 
The  Air  Force  HE  manager  may  also  select  another  contractor  to  perform  the 
HE  effort.  Numerous  small  HFE  companies  provide  complete  HE  services  in 
analysis,  design  criteria,  and  testing.  Although  companies  such  as  MITRE 
and  TRW  do  not  provide  the  complete  HE  effort  as  defined  in  MIL-H-46855, 
they  do  provide  a  thorough  knowledge  of  major  system  acquisition  and  of  HE 
effort  monitoring. 

In  addition  to  the  problems  Of  determining  whether  in  fact  the  Air  Force 
or  other  sources  do  have  a  better  capability  to  provide  a  portion  of  the 
HE  effort,  the  Air  Force  HE  manager  takes  on  the  added  tasks  of 
coordination  between  split  HE  effort  allocations.  This  also  requires  that 
the  proper  budget  is  provided  along  with  the  time  and  personnel  for  the 
lab/test  center  to  do  the  job. 

3.5.2  Contractor  Allocation 

It  Is  an  unusual  situation  th&t  a  major  Air  Force  system  acquisition  con¬ 
tractor  would  allocate  a  complete  HE  effr.it  to  a  subcontractor  or  even  an 
associate  contractor.  However,  the  use  of  consultants,  subcontractors, 
and  associate  contractors  to  perform  portions  of  the  total  HE  program  is 
not  unusual.  Several  competent  consultants  are  available  to  work 
specialized  aspects  of  HFE,  particularly  in  the  biomedical  area.  A  few 
consultants  may  be  helpful  in  the  area  of  automatic  (computer)  design  and 
analysis  techniques. 
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If  a  major  acquisition  contract  Is  split  between  two  or  more  major 
contractors,  and  one  Is  not  designated  as  prime,  an  integrating  agency  or 
contractor  Is  necessary  to  coordinate  the  effort.  The  allocation  of  HE 
effort  should  be  as  described  in  a  plan  developed  by  the  integrating 
agency/contractor.  If  required,  associate  contractor  HE  plans  should  be 
incorporated  In  some  manner  Into  an  integrated  HFE  plan.  This  integrated 
plan  should  describe  the  level  of  effort  each  associate  contractor  must 
maintain.  It  must  describe  the  HE  tasks  (including  task  analysis  formats) 
each  must  perform  and  the  HE  data  outputs  from  those  tasks,  which  will  be 
submitted  to  the  integrator  in  accordance  with  the  HE  program  schedule. 

The  plan  should  be  prepared  in  the  same  manner  as  described  in  Dl-H-7051. 

The  HE  effort  to  be  performed  by  subcontractors  is  proportional  to  the 
size  of  their  contract  and  the  nature  of  their  work.  It  is  primarily  the 
job  of  the  prime  contractor  HP.  manager  to  decide  how  much  HE  the 
subcontractor  shall  perform.  Because  the  prime  contractor  is  always 
responsible  for  the  total  HE  effort,  both  prime  and  sub,  he  may  wish  to 
have  more  of  the  total  effort  done  by  his  organization.  Frequently,  when 
the  requirement  for  MIL-H-4685S,  including  the  HE  plan,  is  levied  on  the 
prime  contractor,  they  will  not  pass  the  requirement  on  to  the  subcontrac¬ 
tor^).  Nearly  always  when  the  requirement  for  MIL-ST0-147?  is  levied, 
this  will  be  passed  on  down  to  the  sub(s).  The  reason  for  this  is  that  It 
is  both  easy  and  cost  effective  to  informally  coordinate  between  a  prime 
and  subcontractor  to  insure  that  HE  methodology  (i.e.,  MIL-H-46855)  is 
performed  correctly.  It  is  extremely  difficult  to  redesign  subcontractor 
equipment  to  incorporate  HE  desiqn  criteria  ( 1 e . ,  MI L-STD-1 47 2)  which  had 
not  been  required  originally.  It  is  easy  and  cost  effective  to  require 
Its  original  application. 

3.6  RFP  Preparation 

Based  on  all  of  the  previously  developed  source  data  and  allocation 
decisions,  the  Air  Force  HE  manager  is  now  able  to  provide  HE  Inputs  to 
the  RFP.  These  inputs  should  generally  be  provioed  to  three  separate 
portions  of  the  RFP.  These  are  the  SOW,  preliminary  system  specification, 
and  the  CORE.  Other  possible  sections  which  may  contain  HE  data  are  the 
proposal  preparation  instructions  and  evaluation  factors  for  award. 
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3.6.1  HE  RFP  Inputs 

Because  this  guide  Is  directed  primarily  toward  major  system  acquisitions, 
the  SOW  should  contain  a  MIL-H-46855  and  MIL-STD-147 2  call  out  under  the 
"Reference  Documents"  section.  It  should  also  contain  words  to  the  effect 
j  that  these  two  documents  are  required  and  the  HE  program  should  be  run  in 

|  accordance  with  their  direction. 

i 

!  The  preliminary  system  specification  should  contain  a  paragraph  (generally 

j  Paragraph  3.3.7,  in  accordance  with  MIL-STD-490,  Ref.  32)  which  calls  out 

i 

/  the  Human  Performance/Human  Engineering  requirement  including  the  specifi¬ 

cation  and  standard.  Depending  on  the  phase  of  the  acquisition,  the 
*  budget,  the  acquisition  strategy,  and  with  the  approval  of  the  Data 

j  Management  Officer  and  the  program  manager,  the  HE  program  manager  may  in- 

!  elude  any  or  all  of  the  Human  Engineering  Data  Items  (see  Table  3.6-1)  in 

the  CDRl. 


TABLE  3.6-1  HUMAN  ENGINEERING  DATA  ITEMS* 

a. 

01 -H -7051 

Human  Engineering  Program  Plan 

b. 

DI-H-7052 

Human  Engineering  Dynamic  Simulation  Plan 

c. 

0I-H-7053 

Human  Engineering  Test  Plan 

d. 

DI-H-7054 

Human  Engineering  System  Analysis  Report 

e. 

0I-H-7055 

Critical  Task  Analysis  Report 

f . 

DI-H-7056 

Human  Engineering  Design  Approach  Document-Operator 

9- 

DI-H-7057 

Human  Engineering  Design  Approach  Document-Maintainer 

h. 

DI-H-7058 

Human  Engineering  Test  Report 

1. 

DI-H-7059 

*A1 1 

Human  Engineering  Progress  Report 

approved  Standard  Data  Items  are  in  DoD  5000. 19L 
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DI-H-7Q51,  Human  Engineering  Program  Plan,  Is  the  most  inclusive  HE  Data 
Item  and  may  be  used  alone.  It  may  be  noted  that  MIL-H-46855  requires  HE 
Program  Planning.  However,  the  only  reasonable  way  to  specify  a  HE  Pro¬ 
gram  Plan  is  to  list  such  a  requirement  (DI-H-7051)  in  the  CORL. 

The  major  portion  of  the  contractor  HE  effort  to  be  performed  should  be 
briefly  described  in  the  SOW.  In  addition  to  the  HE  specification  and 
standard  call  outs,  this  section  should  indicate  the  particular  type  of 
work  the  Air  Force  HE  manager  feels  must  absolutely  be  performed.  This 
may  include  trade-offs  (e.g.,  crew  size),  mock ups  or  simulations.  The 
recommended  methodology  to  be  used  by  the  contractor  may  also  he  indicated 
in  the  SOW.  If  there  are  any  particular  HE  objectives,  such  as  crew  size 
or  performance,  these  should  be  so  stated.  It  is  generally  better  to  in¬ 
clude  all  the  HE  efforts  for  the  program  in  a  single  section  of  the  SOW 
rather  than  apportion  them  to  each  of  the  applicable  subsystems.  The  con¬ 
tractor  should  respond  in  the  same  manner  and  the  total  effort  may  thereby 
be  prepared  and  reviewed  with  less  total  effort. 

Whereas  the  work  to  be  required  in  the  SOW  as  described  by  the  Air  Force 
HE  manager  must  be  within  reasonable  limits  of  what  the  total  contractor 
budget  will  allow,  the  contractor  counterpart  of  the  Air  Force  HE  manager 
would  generally  prefer  being  required  by  the  contract  (or  RFP)  to  do  too 
much  rather  than  too  little.  The  RFP  effort  required  of  the  Air  Force  HE 
manager  is  by  far  the  most  significant  single  factor  in  insuring  an 
adequate  HE  program.  All  program  requirements  must  be  included  in  the  RFP 
package  initially.  The  cost  to  add  requirements  at  a  later  date  with  an 
engineering  change  proposal  (ECP)  is  generally  prohibitive  in  comparison 
to  the  gain  from  the  added  contractor  responsibility  for  an  additional 
task. 

Ouring  the  RFP  preparation,  a  source  selection  plan  should  be  prepared  by 
the  project  office.  Included  in  the  plan  and  the  RFP  should  be  evaluation 
criteria  and  standards  which  indicate  a  proposal  score  given,  in  part,  for 
the  contractor  HE  effort.  This  score  allocation  to  HE  should  insure  a 
best  effort  contractor  response  to  the  HE  aspects  of  the  RFP. 
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3.6.2  Tailoring 

During  the  past  few  years,  the  subject  of  specification  tailoring  has 
gained  popularity;  presumably  because  of  DoO  Directives  and  the  well  known 
0MB  A-109  circular  describing  major  system  acquisition  methods.  The  gen¬ 
eral  notion  of  specification  tailoring  is  based  on  the  concept  that  the 
reason  many  systems  acquisitions  cost  so  much  is  that  they  are  designed 
and  buiit  per  specifications  which  require  design  constraints  which  In 
many  cases  are  not  really  'useful*  or  appropriate  either  to  that 
particular  program  or  for  a  particular  design  phase.  Tailoring  is  an 
attempt  to  modify  specifications  to  require  only  that  which  Is  useful  to 
the  planned  system  acquisition  phase. 

Whereas  there  is  little  question  as  to  the  short  term  cost  effectiveness 
of  HF  specification  tailoring,  there  is  serious  doubt  as  to  the  effects  of 
this  tailoring  on  life  cycle  costs  (LCC's).  Before  tailoring  is  accom¬ 
plished  by  either  the  Air  Force  or  contractor,  a  few  extremely  significant 
factors  must  be  considered: 

a)  The  probability  that  the  program  will  complete  the  full  acquisi¬ 
tion  cycle. 

b)  The  nature  of  the  specification  tailoring  savings  as  short  term 
only,  long  term  only,  or  both  short  and  long  term. 

c)  The  amount  of  short  term  savings  due  to  tailoring. 

d)  The  cost  to  change  the  system  design  to  meet  long  term  system 
performance  requirements  (e.g.,  maintainability  and  operability) 
not  necessary  for  the  initial  acquisition  phases. 

e)  The  probable  increased  life  cycle  costs  associated  with  waiving 
the  reliability,  maintenance,  and  operability  requirements 
normally  specified  for  an  operationally  deployed  system. 

f)  The  comparison  between  items  c),  d)  and  e)  above. 

The  answer  to  the  first  factor  Is  "most  probable”.  Very  few  programs  ever 
fail  to  pass  their  Defense  System  Acquisition  Review  Council  (DSARC) 
milestone  review  meetings.  Therefore,  both  long  term  (LCC)  and  short  term 
savings  are  significant.  If  the  savings  are  short  term  only,  they  need  to 
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be  balanced  against  possible  increased  life  cycle  costs  that  they  could 
cause.  These  costs  could  be  for  engineering  c'nrne  proposals  (ECP’s), 
system  design  revisions,  operator  or  malntalne-  errors  resulting  in  costly 
failures,  equipment  malfunctions,  or  safety  hazards. 

Rather  than  recommend  tailoring  of  MIL-H-46855  and  particularly  MIL-STD- 
1472  during  the  RPR  preparation,  it  is  recommended  that  any  tailoring  to 
be  performed,  if  any,  be  accomplished  by  the  contractor  in  his  proposal 
response  (to  the  RFP).  The  Air  Force  should  specify  both  specifications 
as  is  and  invite  tailoring  of  them  in  accordance  with  DI-H-7051  (HE  Plan). 

The  total  notion  of  tailoring  as  applied  to  HE  is  somewhat  ironic  in  that 
MIL-H-46855  has  always  clearly  stated  that  it  may  be  Invoked  on  contracts 
either  in  its  entirety  or  selectively.  In  his  HE  Plan,  the  contractor  has 
always  described  those  HE  tasks  which  he  determines  are  most  cost  effec¬ 
tive  to  perform.  In  accordance  with  MIL-H-46855  and  the  HE  Plan  Data  Item 
(DI-H-7051),  the  contractor  provides  what  he  feels  is  a  tailored  version 
of  the  HE  tasks  to  be  accomplished  (or  not  to  be  accomplished)  for  the 
proqram. 

If  the  program  is  not  a  major  acquisition,  the  Air  Force  HE  manager  should 
determine  the  general  applicability  of  MIL-H-46855  to  the  particular 
program.  The  suggested  method  of  doing  this  is  included  as  an  appendix  to 
MIL-H-46855.  It  should  be  noted  that  if  users  of  this  guide  are  working 
with  major  systems  acquisitions,  MIL-H-46855  should  be  applied. 

It  should  be  further  noted  that  if  an  Air  Force  HE  manager  has  already 
been  assigned  to  a  program,  the  chances  are  high  that  MIL-H-46855  should 
also  be  required.  Conversely,  if  MIL-H-46855  is  not  required,  the  need 
for  the  HE  manager  is  questionable. 

In  a  somewhat  similar  manner,  the  possible  need  to  tailor  MI L-Sl D-1472  is 
superfluous.  The  application  of  the  standard  in  its  entirety  to  a  program 
costs  little  if  applied  early.  The  few  situations  which  might  arise  that 
would  cause  a  high  system  cost  or  performance  decrement  as  a  result  of 
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application  of  the  criteria  are  easily  solved.  Each  of  these  deviations 
from  MIL-STO-1472  design  criteria  are  presented  during  design  reviews  to 
‘he  Air  Force  customer  on  a  design  criteria  deviation  request  sheet.  This 
deviation  request  may  or  may  not  be  a  part  of  ar.  existing  engineering 
change  proposal  (ECP).  The  deviation,  cause,  and  implications  are 
summarized  and  if  approved,  which  they  generally  are,  signed  off  by  the 
Air  Force.  If  not  approved,  the  criterion  is  incorporated  into  the 
design,  along  with  the  increased  system  costs  required  to  implement  the 
design  requirements. 

Where  MI L-STO-1 472  criteria  are  clearly  not  applicable  due  to  absence  of 
the  particular  hardware  or  system  functions  for  which  the  criteria  were 
intended,  it  is  not  necessary  to  call  out  all  the  exceptions.  This  task 
is  generally  too  tedious  to  be  of  value.  The  error  of  omission  in  not 
calling  out  the  application  of  pertinent  criteria  is  more  serious  than  the 
error  of  commission,  calling  out  criteria  which  would  apply  to  nonexistent 
hardware  or  system  functions.  The  latter  mistake  is  easily  forgiven  and. 
If  necessary,  can  be  provided  for  in  the  above  described  criteria 
deviation  request  forms  wnich  should  be  described  in  the  contractor’s  HE 
program  plan.  The  error  of  omitting  the  requirement  for  appropriate 
design  criteria  could  easily  lead  to  a  costly  engineering  change  proposal 
(ECP),  or  worse  yet,  to  iqnoring  the  needed  criteria  and  risking  the 
consequences  of  the  degraded  man-machine  performance. 

As  previously  indicated,  the  most  fruitful  area  to  perform  HE  tailoring  is 
in  the  HE  program  Plan  (DI-H-7051).  Other  data  items  may  also  he 
tailored.  The  tailoring  of  the  data  items  should  correspond  to  the 
tailoring  of  MIl-H-46855  and  its  requirements  on  the  contractor.  The  data 
items  may  be  omitted  if  not  necessary  or  they  may  be  modified  to  delete 
any  Ineffective  or  costly  portions  which  do  not  apply  to  the  particular 
program. 
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3.6.3  Or  aft  RFP's 


Frequently,  in  order  to  create  a  better  quality  RFP,  a  draft  RFP  is  issued 
to  potential  competitors  for  their  review  and  comment.  Such  drafts  have 
advantages  in  that  the  Air  Force  can  try  out  requests  for  particular  pro- 
gram  tasks,  provisions,  or  methodologies.  Industry  feedback  on  draft  RFPs 
has  the  potential  for  effecting  substantial  savings  by  pointing  out 
unnecessary  constraints.  The  contractor's  responses  to  the  final  RFP  are 
generally  of  better  quality  since  they  have  had  more  time  to  work  the 
requested  proposal  problem.  The  Air  Force  HE  manager  should  participate 
in  the  draft  review  in  order  to  suggest  the  kind  of  effort  that  he  feels 
should  be  requested  in  the  RFP. 

3.7  Proposal  Preparat  jn 

If  the  Air  Force  has  issued  a  draft  RFP,  the  contractor  responds  by 
providing  a  critique  and  suggestions.  The  contractor  is  aware  of  the 
total  problem  and  should  produce  a  better  quality  proposal.  The  contrac¬ 
tor  HE  manager  should  participate  in  the  draft  review  in  order  to  suggest 
the  kind  of  effort  that  he  feels  should  be  requested  in  the  RFP. 

Once  the  RFP  is  officially  issued,  the  decision  as  to  how  to  respond  is 
invariably  made  by  the  contractor  program  manager  within  the  limitations 
of  the  proposal  evaluation  criteria  supplied  with  the  RFP.  The  program 
manager  may  simply  choose  to  respond  in  kind  to  each  of  the  requested 
tasks  listed  in  the  RFP  statement  of  work  (SOW).  As  a  minimum,  the  con¬ 
tractor  must  state  agreement  with  the  SOW  and/or  take  exception  to  those 
portions  he  does  not  wish  to  comply  with.  The  contractor  should  also 
indicate  his  acceptance  of  the  CORE  item.  Frequently,  this  means 
providing  a  preliminary  copy  of  the  HE  Plan  in  accordance  with  MIL-H-46855 
and  DI-H-7051.  If  the  Preliminary  HE  Plan  is  requireu,  most  of  the 
proposed  HE  effort  may  be  contained  In  the  plan.  If  the  plan  is  not 
required,  the  HE  effort  should  be  described  in  the  technical  portion  of 
the  proposal.  In  some  cases,  an  HE  Plan  is  submitted  although  not  re¬ 
quired  per  the  RFP.  In  any  case,  the  following  subjects  should  be  includ¬ 
ed  in  the  Plan  or  the  HE  portion  of  the  proposal: 
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a)  Procedures  that  are  proposed  for  complying  with  MIt-H-46855 
requirements.  This  includes  anticipated  trade  studies  and 
analysis,  design,  evaluation  techniques  Intended  to  be  provided. 

b)  The  company's  organizational  elements  and  (If  possible)  personnel 
selected  to  Implement  the  HE  program. 

c)  The  HE  efforts  accomplished  (and  lessons  learned)  during  previous 
program  phases  should  be  summarized. 

d)  The  proposed  HE  participation  In  simulation,  mockups,  equipment 
detail  design,  testing,  and  verification  should  be  described. 

e)  Special  HE  objectives  (e.g.,  crew  size  and  performance)  and  anti¬ 
cipated  problems  should  be  included  along  with  the  proposed  means 
to  meet  these  objectives  and  solve  these  problems. 

f)  A  time-phased  edule  showing  initiation  and  completion  dates  of 
significant  lestones. 

When  the  plan  is  used  to  describe  the  HE  effort,  this  effort  should  be  an 
integral  part  of  the  total  program  management  and  engineering  effort.  The 
plan  should  include  details  of  the  Implementation  of  each  tasx  identified 
by  the  tailored  application  of  MJl-H-46855.  The  plan  should  describe  the 
requirements  for  HE  management  required  to  support  the  program  through  the 
total  period  of  the  contract.  The  olan  should  detail  the  HE  interfaces 
with  all  levels  of  program  management.  It  should  show  clear  evidence  of 
specification  tailoring  consideration  and  of  design  to  cost  and  design  to 
life  cycle  costs.  The  cost  of  imposing  HE  requirements  should  be  evalua¬ 
ted  against  the  benefits  that  will  be  realized. 

After  the  issuance  of  the  RFP  and  before  the  contract  award,  the  Air  Force 
evaluators  are  no  longer  free  to  converse  with  prospective  contractors  on 
an  informal  day-to-day  basis.  From  that  point  on,  everything  will  be 
documented  and  coordinated  through  the  appropriate  contracting  officer. 


3.0  Proposal  Evaluation 

The  Air  Force  HE  manager  should  play  an  active  role  In  the  customer 
proposal  evduatlon  process.  He  must  participate  as  a  member  of  the 
source  selection  team  to  Insure  that  the  contractor's  Intended  approach  to 
dealing  with  the  system  man-machine  interface  meets  the  criteria  described 
in  the  previous  Section  (3.7).  The  Air  Force  HE  manager  must  develop  the 
facts  upon  which  to  base  source  section.  He  must  be  able  to  determine 
whether  the  potential  contractor  understands  what  needs  to  be  done.  This 
includes  understanding  of  the  HE  requirements  and  scope  and  magnitude  of 
the  project,  realism  of  approach,  nsk  assessment,  and  life  cycle  cost 
Implications.  The  HE  contractor  must  clearly  show  that  the  requirements 
are  recognized,  that  a  preliminary  analysis  was  made  in  arriving  at  the 
approach,  and  that  the  requirements  will  be  satisfied  in  a  timely  and 
cost-effective  manner.  The  areas  in  which  trade-off  decisions  will  need 
to  be  made  should  be  identified  with  candidate  alternatives  and  the 
rationale  and  schedule  for  their  selection.  The  Air  Force  HE  manager  must 
check  to  insure  intended  compliance  with  SOW,  system  specification,  and 
CORL  requirements.  If  a  preliminary  HE  Program  Plan  is  called  for,  much 
of  the  evaluation  can  be  made  by  a  thorough  review  of  the  plan.  Evalua¬ 
tion  ratings  and  rankings  must  be  in  accordance  with  the  overall  source 
selection  plan  established  for  the  system. 

The  contractor's  directly  applicable  and  related  HE  experience  should  be 
evaluated.  The  contractor  must  clearly  Indicate  the  relevance  of  experi¬ 
ence  gained  in  similar  programs  of  equal  or  greater  complexity.  The  con¬ 
tractor  may  wish  to  provide  "lessons  learned"  and  to  show  how  his  experi¬ 
ence  will  benefit  the  particular  proposed  program.  The  relation  of  HE  to 
other  disciplines  must  be  indicated  as  well  as  the  relation  of  HE  to  pro¬ 
gram  management.  However,  the  later  relationship  should  not  be  evaluated 
as  being  right  or  wrong  but  should  be  presented  to  the  Air  Force  customer 
for  his  information.  Consideration  of  design  to  cost  or  design  to  life 
cycle  cost  as  it  affects  HE  should  also  be  evident  In  the  proposal. 
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3.9  Contractor  Task  Accomplishment 

After  the  award  of  the  contract,  the  major  portion  of  the  program  effort 
is  in  the  hands  of  the  contractor.  Along  with  several  other  technologies, 
HE  must  refine  its  program  planning  and  scheduling  ef '  •*■..  It  must  ini¬ 
tiate  the  development  of  system  requirements,  cond-'t  .  vrade 
studies,  participate  in  the  design  of  the  program  deve.^^.  'el,  and 

evaluate  the  design  model  through  the  use  of  appropriate  tesc  i.  <iques. 

3.9.1  Meetings 

Within  a  few  weeks  of  the  contract  award,  a  guidance  meeting  should  be 
arranged  between  the  Air  Force  and  contractor.  The  purpose  of  this 
meeting  is  for  a  face  to  face  discussion  of  what  each  of  the  two  parties 
feels  is  the  necessary  HE  (or  HFE)  effort  for  the  program.  The  Air  Force 
should  tell  the  contractor  his  evaluation  of  the  HE  inputs  to  the 
proposal.  If  an  HE  Plan  was  submitted,  this  evaluation  will  be  directed 
primarily  to  that  item.  The  Air  Force  HE  monitor  should  provide  the  con¬ 
tractor  with  detailed  guidance  as  to  the  problems  and  the  needs  the  HE  ef¬ 
fort  should  address.  The  meeting  may  be  used  to  discuss  customer  sources 
of  analysis  input  data  not  previously  known  to  the  contractor.  The  con¬ 
tractor  choice  of  analysis,  design,  and  test  techniques  may  be  reviewed. 
Significant  human  performance  requirements  should  be  defined  to  avoid 
later  misunderstanding. 

HE  will  also  participate  in  program  design  reviews  such  as  the  PDR  &  COR. 
Results  of  HE  efforts,  including  applicable  trade  studies  and  critical 
task  analysis,  will  oe  reported.  Derived  HE  design  criteria  and  applica¬ 
ble  HE  design  requirements  should  be  presented. 

3.9.2  Detailed  HE  Plan 

If  a  Preliminary  HE  Plan  was  required  as  a  part  of  the  total  program 
proposal,  a  Detailed  HE  Plan  should  be  prepared  subsequent  to  the 
customer-contractor  guidance  meeting  and  submitted  as  a  part  of  the 
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Proqram  Plan  for  the  system  (Ref.  DI-H-7051).  When  this  plan  Is  approved 
by  the  SPO,  it  may  be  used  by  the  contractor  to  direct  his  program 
efforts.  If  any  changes  to  these  efforts  (as  described  in  the  plan) 
occur,  the  contractor  must  report  and  justify  them  to  the  SPO. 

3.9.3  Basic  Considerations 

Previous  sections  of  this  guide  indicated  the  importance  of  MIl-H-46855  to 
the  accomplishment  of  the  HE  effort.  It  is  the  purpose  of  this  section  to 
briefly  present  basic  considerations  not  covered  in  the  MIl-H-46855  re¬ 
quirements  or  other  data  presented  in  Section  3.1.  These  considerations 
consist  of  the  type  of  data  required  to  start  any  HE  effort,  when  to  per¬ 
form  the  effort,  the  level  of  detail  required,  and  the  type  of  specific 
results  normally  expected  from  the  HE  effort.  Later  paragraphs  of  this 
guide  deal  with  these  basic  considerations  in  relation  to  specific  HE 
techniques,  but  this  paragraph  pertains  to  these  basic  considerations  in 
relation  to  the  overall  HE  effort. 

3.9. 3.1  Data  Inputs 

There  is  a  large  variation  in  the  degree  to  which  data  inputs  such  as  mis¬ 
sion  requirements,  system  requirements,  or  operational  concepts  will  be 
supplied  by  the  customer  or  by  contractor  program  organization  other  than 
HE.  More  often  than  not,  mission  analysis  and  functional  flow  diagrams 
are  not  provided  to  the  HE  group.  The  current  tendency  in  industry  is  to 
have  this  type  of  information  generated  by  HE.  Other  technologies  such  as 
software  design  and  displays/controls  provide  data  to  HE  as  to  the  soft¬ 
ware  and  hardware  capabilities  and  limitations.  Data  inputs  pertaining  to 
human  performance  and  previous  system  experience  have  to  come  from 
research,  literature,  or  from  personnel  experience.  The  specific  data 
sources  for  these  inputs  are  either  too  numerous  or  too  intangible  to  list 
here.  The  data  inputs  for  the  later  design  and  test  phases  of  HE  are 
obtained  from  HE  analysis  or  from  other  technologies. 
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3. 9. 3.2  Timing 

Without  the  proper  scheduling  of  the  HE  analysis,  design,  and  testing 
effort,  it  can  turn  out  to  be  of  little  use  to  the  system  design.  It  Is 
not  sufficient  just  to  perform  these  HE  efforts.  It  is  equally  important 
to  demonstrate  that  the  results  of  the  effort  will  be  completed  or  par¬ 
tially  completed  at  a  point  In  the  schedule  when  It  can  properly  impact 
the  system  design.  Occasionally,  the  HE  efforts  are  performed  on  a  por¬ 
tion  of  a  program  that  later  evolves  to  the  point  where  the  HE  effort  must 
be  performed  again  to  be  pertinent.  Sometimes  the  results  of  the  effort 
are  premature  to  their  use  by  other  technologies.  However,  all  too  often 
HE  tasks  are  performed  as  an  after-the-fact  documentation  exercise  or  just 
a  workaround  procedure  that  appears  in  a  technical  publication.  The  later 
the  analysis,  design,  or  test  is  performed,  the  less  chance  there  is  to 
impact  the  crew  station  or  other  man/machine  interface.  Late  findings  of 
serious  crew  system  problems  can  be  extremely  expensive  in  redesign  and  in 
retraining,  or  worse  yet,  late  inputs  may  be  disregarded  to  the  extent  of 
causing  serious  system  failures  and  accidents. 

3. 9. 3. 3  Level  of  Detail 

Ju:t  as  the  HE  effort  may  be  performed  too  soon  or  too  late,  the  analysis, 
design,  cr  testing  detail  can  be  performed  at  too  gross  or  too  detailed  a 
level.  A  discussion  of  the  definition  of  various  levels  of  analysis  is 
contained  in  a  later  paragraph.  The  level  of  analytical  detail  that 
should  be  performed  is  significant  to  the  HE  manpower  effort.  Analysis 
must  be  performed  judiciously  to  insure  that  proper  emphasis  is  given  to 
each  of  the  various  tasks  or  mission  functions  which  are  candidates  for  HE 
analysis.  It  is  the  job  of  the  human  engineer  or  HE  manager  to  decide 
which  level  of  analysis  will  lead  to  worthwhile  data  or  useful  design 
criteria.  For  example,  new  system  designs  or  programs  often  contain  func¬ 
tional  requirements  that  are  identical  to  previously  designed  and  tested 
systems.  There  is  no  point  in  repeating  a  detailed  analysis,  design,  or 
test  that  has  already  been  accomplished.  It  is  simply  not  cost  effective, 
especially  when  new  program  schedules  and  manpower  budgets  generally  are 
extremely  limited. 
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The  level  of  analytical  detail  achieved  djring  functional  allocation 
trades  must  suffice  to  permit  positive  allocation  of  functions  to 
operators,  equipment,  or  software.  The  functional  allocation  analyses 
have  not  been  performed  satisfactorily  If  the  answers  to  the  trades  tend 
to  come  out  as  a  combination  of  operator /equipment/software  allocations. 
More  detailed  tas<  analysis  should  be  performed  only  on  critical  tasks  or 
in  accordance  with  required  Data  Item  Descriptions.  In  similar  manner  for 
design.  If  other  organizations  have  the  charter  to  perform  the  detailed 
design  of  program  hardware,  it  behooves  HE  personnel  to  provide  little 
more  than  the  HE  design  criteria.  The  details  of  the  complete  design,  in¬ 
cluding  specifications  and  drawings  should  not  be  performed  by  HE.  On  the 
other  hand,  HE  personnel  cannot  offer  just  negative  criticism  of  other 
organizations'  designs.  All  criticism  must  include  sufficient  detail  to 
let  the  designer  know  specifically  what  is  wrong  with  the  design  and  what 
could  be  done  to  modify  it  to  meet  proper  HE  design  criteria.  It  is  also 
the  job  of  the  HE  T&E  observer  or  manager  to  decide  what  level  of  T&E  will 
lead  to  worthwhile  data  or  useful  design  criteria.  For  example,  there  is 
no  need  to  examine  new  system  portions  which  are  identical  to  satisfactory 
old  systems.  On  new  system  designs,  it  may  be  necessary  to  examine  data 
down  to  as  much  detail  as  a  tenth  of  a  second.  If  the  HF  program  has  been 
properly  managed,  all  system  potentially  critical  tasks  will  have  been 
previously  indicated  for  special  HE  T&E  considerations.  In  any  case,  the 
need  to  gather  human  performance  related  T&E  field  data  more  accurately 
than  a  tenth  of  a  second  is  extremely  doubtful.  In  a  similar  manner,  the 
HE  observer  should  maintain  adherence  to  the  rules  for  significant  figures 
and  common  sense  when  gathering  data  on  light  levels,  sound  levels,  reach 
envelope  measurements,  etc. 

3. 9. 3. 4  Applications 

The  purpose  of  performing  the  three  major  HE  activities  (analysis,  design 
and  test)  is  to  help  develop  and  justify  a  system  design.  The  purpose  of 
doing  HE  analysis  to  successively  detailed  levels  is  to  "drive  out"  or 
identify  more  and  more  significant  detailed  design  requirements.  Examples 
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of  such  data  are:  how  many  and  what  kinds  of  personnel  will  use  the 
system;  what  the  crew  performance  limits  are  in  terms  of  time,  space, 
force  and  reliability;  and  what  the  possible  alternative  solutions  are. 
Design  requirements  are  incorporated  into  mockups,  drawings,  and 
specifications.  The  end  product  of  HE  T&E  is  to  verify  system  design, 
discover  system  inadequacies,  and  provide  recommendations  for  design  or 
other  system  changes.  In  addition,  a  by-product  may  be  to  provide  infor¬ 
mation  for  a  data  bank  of  human  performance  and  crew  systems  design 
related  data  to  be  used  on  later  programs.  Generally,  the  outputs  of 
these  efforts  should  be  condensed  and  otherwise  modified  to  make  them  more 
easily  understood  by  the  program  personnel  who  have  use  of  them  and  are 
not  trained  in  HE  techniques.  Tables  3.9-1,  -2  and  -3  show  most  of  the 
applications  for  data  developed  from  using  the  various  listed  techniques. 

It  may  be  useful  for  the  applications  or  specific  output  data  to  be 
prioritized  in  some  manner  to  show  that  there  are  certain  absolutely 
essential  system  HE  design  requirements  or  modifications  which  are 
necessary.  The  risk  of  not  doing  this  is  to  have  insignificant  results 
acted  upon  and  critical  data  ignored.  All  findings  must  oe  well  documen¬ 
ted  and  files  must  be  maintained.  By  themselves,  verbal  inputs  (HE 
outputs)  as  to  analysis,  design,  or  T&E  results  have  virtually  no  chance 
of  acceptance. 

3.9.4  Analysis 

In  order  to  develop  HE  performance  criteria  and  hardware  HE  design  crite¬ 
ria  and  to  accomplish  the  required  analysis  described  in  MIl-H-46855,  a 
concerted  analysis  effort  must  be  accomplished. 

Initial  development  of  man-machine  interface  concepts  must  be  concurrent 
with  advanced  development  of  system  concepts.  During  this  formative 
period  of  system  development,  the  human  engineer  has  a  number  of  important 
responsibilities? 


Table  3.9- 1.  Human  Engineering  Analysis  Techniques  Data  Applications 


Table  3.9-2.  Human  Engineering  Design  Techniques  Data  Applications 
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a)  Assurance  that  human  engineering  Inputs  are  Incorporated  Into 
system  design  requirements  documentation; 

b)  Major  participation  in  the  allocation  of  system  functions  to  man, 
machine,  or  software,  or  combinations  thereof; 

c)  Assurance  that  each  candidate  system  functional  implementation  is 
feasible  in  all  respects  from  a  human  engineering  standpoint; 

d)  Development  of  design  concepts  for  each  operator/malntainer  work 
station  to  the  point  that  it  is  reasonably  assured  such  a  work 
station  arrangement  is  operable; 

e)  Performance  and  documentation  of  preliminary  hardware  trade 
studies  pertaining  to  human  engineering  considerations; 

f)  Identification  of  potential  human  engineering  problem  areas  which 
may  require  attention; 

g)  Preparation  of  inputs  to  subcontractor  RFP  packages  as 
applicable. 

These  tasks  are  frequently  accomplished  by  the  analysis  process  of 
breaking  them  down  into  smaller  and  smaller  elements  to  the  point  where 
they  can  be  handled.  At  the  smaller  element  level,  significant  aspects  of 
the  total  problem  can  be  examined  in  detail.  Answers  to  several  detailed 
questions/problems  are  more  easily  obtained  than  answers  to  a  few  top 
level  question/problems. 

Generally,  the  analysis  process  starts  with  the  system  mission  as 
described  by  a  baseline  scenario.  The  mission  objective  and  functions 
that  must  be  performed  by  the  system  are  identified,  described,  and 
sequenced.  These  functions  are  then  analyzed  to  determine  their  proper 
allocation  to  personnel,  software,  or  equipment,  or  some  combination  of 
these.  Once  allocated,  the  personnel  functions  are  further  analyzed  to 
determine  the  specific  operator/maintainer  tasks  which  must  be  performed 
to  accomplish  the  functions.  The  tasks  are  further  detailed  to  show 
estimated  time  and  space  relationships. 


Over  the  years,  human  engineers  have  developed  a  number  of  powerful  tools 
and  techniques  to  aid  in  applied  human  engineering  work.  Each  of  the  fol¬ 
lowing  subparagraphs  describes  the  characteristics  of  one  technique.  In¬ 
formation  is  supplied  as  to  what  the  technique  is,  what  it  is  Intended  to 
do,  and  why  it  is  useful.  Much  of  this  Information  is  presented  in 
tabular  form  in  Table  3.9-4.  By  listing  each  of  the  techniques  on  one 
table,  they  may  be  more  easily  compared  for  selection  and  use.  An  expla¬ 
nation  of  each  of  the  table  Selection  Evaluation  Characteristics  is  pro¬ 
vided  in  Table  3.9-5.  Procedures  for  the  construction  of  each  technique 
are  provided.  When  significant,  the  limitations  as  to  what  the  technique 
will  not  do  are  pointed  out.  Also  included  are  sample  formats  to 
illustrate  the  layout  and  details  of  several  of  the  techniques. 

This  guide  contains  only  the  better  known  techniques.  Reference  11  (Geer, 
1976)  contains  data  on  a  few  additional  techniques.  If  for  some  reason  it 
is  felt  tnat  existing  techniques  will  not  accomplish  the  required  analysis 
task,  then  obviously  new  techniques  should  be  developed.  The  development 
of  new  paper  and  pencil  analysis  techniques  is  generally  not  difficult. 

The  major  drawback  in  doing  this  is  the  extra  educational  process  that  is 
required  to  assist  those  wishing  to  understand,  review,  or  otherwise  use 
the  analysis. 

3.9.4. 1  Mission  Profiles 

Description:  Mission  analysis  is  the  first  step  in  the  system  develop¬ 

ment  required  for  the  establishment  of  human  factors  design 
criteria.  The  system  mission  or  operational  reauirements 
are  a  composite  of  requirements  starting  with  general  oper¬ 
ational  requirements  and  progressing  through  specific  oper¬ 
ational  requirements.  The  mission  requirements  define  the 
system  in  terms  of  limits  of  operation  necessary  for 
fulfilling  the  weapons  system  mission  activities.  Mission 
profiles,  along  with  scenarios,  are  the  two  major  tech¬ 
niques  used  to  perform  mission  or  operations  analysis.  The 
total  analysis  process  must  start  with  mission  profiles 
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Table  3.9-4.  Analysis  Techniques  Selection  Chart 
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Table  3.9-5:  Explanation  of  Selection  Evaluation  Characteristics 

Across  the  top  of  Table  3.9-4,  Analysis  Techniques  Selection  Chart,  are  a 
number  of  selection  evaluation  characteristics.  The  purpose  of  this  char¬ 
acteristics  list  is  to  make  evaluative  comments  as  a  part  of  a  tradeoff 
analysis  between  the  various  listed  analysis  techniques.  Some  techniques 
are  obviously  better  than  others  for  certain  types  of  programs,  program 
stages,  or  analysis  efforts.  The  following  list  describes  in  detail  what 
is  meant  by  each  of  the  evaluation  char  - '►eristics. 

DEFINITION  OF  TECHNIQUES  SELECTION  EVALUATION  CHARACTERISTICS 

MOST  APPLICABLE  PROGRAM  STAGE 

The  phase  of  a  program  that  is  best  suiteo  to  the  use  of  this  technique: 
Conceptual  Phase,  Validation  Phase,  Full-Scale  Development  Phase,  and 
Production  Phase  (Ref.  AFM  11-1),  (See  Table  3-4-1). 

RELATIVE  COMPLEXITY 

The  category  of  relative  complexity  that  best  describes  this  technique 
when  compared  to  other  techniques. 

USED  FOR 

The  category  that  best  describes  the  level  of  detailed  analysis  for  which 
this  technique  m3y  b^  used- 

BREADTH 

Indicates  the  relative  quantity  of  different  tasks  that  may  be 
simultaneously  handled  by  using  this  analysis  technique. 

RELATIVE  TIME  TO  PERFORM 

The  time  category  that  best  describes  the  time  to  perform  this  technique 
for  a  given  task,  when  compared  to  other  techniques. 

USED  BY 

Tne  types  of  users  of  the  analysis  technique,  either  or  both. 

RELATIVE  COST 

The  category  that  best  describes  the  relative  cost  of  this  technique  when 
compared  to  other  techniques. 

RELA1 IVE  COST  EFFi  .7- NESS 

The  category  that  b?  *  indicates  hf'*  <  U;ti\e  r  s  t<  unique  is  when 
compared  with  oth  r  techniques. 
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Procedure: 


Use/Validity: 


because  the  human  factors  engineer  must  have  a  good  idea  of 
the  operational  situation  or  events  that  will  be 
confronting  operators  and  maintenance  personnel  in  newly 
conceived  systems.  Although  historically  mission  analysis 
has  been  performed  by  groups  other  than  human  factors,  the 
trend  has  been  either  to  cut  down  on  the  size  of  the  System 
Engineering  or  Operations  Analysis  groups  who  have  done 
this  work,  or  to  eliminate  the  groups  altogether  in  the 
name  of  cost  savings.  However,  the  need  for  the  analysis 
is  as  critical  as  ever  and  the  work  must  therefore  be  per¬ 
formed  by  someone,  such  as  a  human  factors  engineer. 

The  procedure  for  constructing  mission  profiles  is  easy  to 
follow.  The  term  mission  profile  derives  its  name  from  the 
typical  side  view  format  illustrated  in  Figure  3.9-1.  The 
profile  is  a  plot  Of  the  aircraft  flight  in  terms  of  total 
distance  traveled  (or  time)  from  home  base.  Significant 
mission  events  or  functions  are  noted  on  the  plot.  Mission 
"profiles"  other  than  the  illustrated  example  are  also  used 
to  indicate  the  flight  path  in  terms  of  latitude  and  longi¬ 
tude  such  as  would  be  observed  in  a  plan  view  in  a  manner 
similar  to  a  horizontal  situation  display.  These  particu¬ 
lar  riots  are  often  referred  to  as  graphic  scenarios.  Sig¬ 
nificant  aircraft  functions  are  plotted  along  the  route  at 
the  points  of  their  planned  occurrence.  Each  function  de¬ 
scribes  a  clearly  distinguishable  start  and  completion 
point  for  a  mission  segment. 

Along  with  the  Initiation  of  new  programs,  there  is  invari¬ 
ably  the  issuance* of  top  level  program  objectives  and  sys¬ 
tems  operational  requirements,  it  is  a  combination  of 
these  objectives  and  requirements  with  th»  past  experience 
of  previous  similar  systems  which  combine  to  create  the 
mission  profile  data.  If  all  essential  operational 
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requirements  cannot  be  logically  and  realistically  included 
Intn  one  profile,  then  others  must  be  developed  to  cover 
all  functions  In  a  reasonable  context.  Although  mission 
scenarios  are  sometimes  developed  before  mission  profiles, 
they  generally  follow  the  profiles  and  use  the  mission 
profile  functions  to  interact  with  scenario  threat  and 
other  event  data.  In  addition  to  feeding  Into  the 
scenarios,  the  mission  profile  data  are  used  in  the  devel¬ 
opment  of  the  functional  flows.  Table  3.9-1  illustrates 
several  output  applications  that  apply  to  mission  profiles. 

The  inherent  characteristics  of  the  mission  profile  analy¬ 
sis  technique  when  compared  to  other  human  factors  engi¬ 
neering  techniques  are  summarized  in  Table  3.9-4.  Mission 
profiles  should  be  developed  as  early  as  possible  in  the 
program  schedule.  Given  any  sort  of  basic  system  require¬ 
ments  to  build  on,  they  may  be  simple  to  construct. 

However,  if  numerous  threats/events  are  used,  they  become 
much  more  complex.  They  are  generally  used  for  a  gross 
analysis  only.  At  any  one  point  in  time,  they  may  be  used 
to  show  single  top  level  tasks  better  than  several  simulta¬ 
neous  tasks.  Mission  profiles  require  a  minimum  of  time  to 
develop.  They  are  used  equally  by  managers  and  analysts. 
Their  cost  is  low  and  their  cost  effectiveness  ' 
relatively  high. 

3. 9. 4. 3  Mission  Scenarios 

Description:  Scenarios  are  developed  from  the  th'-eat/concept  and  the 

mission  profiles,  and  they  must  fully  describe  the  events 
implied  by  the  profile.  Rather  than  using  a  special  format 
for  scenarios,  they  are  generally  written  in  straightfor¬ 
ward  narrative.  This  narrative  should  c.escribe  the 
proposed  mission  in  detail,  identifying  key  events  and 
Implied  requirements  that  might  otherwise  be  overlooked. 
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Procedure: 


This  Includes  all  essential  system  functions,  such  as 
failure  modes  or  emergency  procedures.  Elements  of  the 
scenario  should  be  sufficiently  detailed  to  convey  an 
understanding  of  the  mission,  and  to  permit  a  breakout  of 
mission  variations  relating  to  features  such  as  a)  mission 
phases,  b)  the  activity  performed  In  each  phase,  c)  tne  ap¬ 
proximate  degree  of  accuracy  for  each  activity,  and  d)  any 
Interdependencies  of  activities  as  to  timing,  information 
transfer,  etc. 

There  are  no  precise  rules  for  writing  scenarios;  however, 
there  are  a  number  of  factors  that  should  be  considered  for 
inclusion  in  them.  These  factors  are: 

a)  Assumed  operational  tactics. 

b)  A  listing  of  subsystems  and  their  proposed  capa¬ 
bilities  (e.g.,  sensor  range,  navigation  accuracy, 
etc.) 

c)  Postulation  of  a  geographic  position  -  this  would 
include  boundaries  and  terrain  elevations. 

d)  A  selected  starting  point  in  terms  of  time  and 
location. 

e)  PI acement  of  both  threats  and  unknowns  within  the 
geographic  area. 

f)  Adherence  to  the  previously  developed  mission 
profile(s)  in  terms,  routes,  and  distance. 

g)  Development  of  limited  profiles  for  each  of  the 
unknown  and  hostile  tracks  (contacts). 

h)  Determination  of  the  location  of  stationary 
threats/targets. 

i)  Based  on  subsystem  capabilities,  determination  of 
when  sensors  are  active  and  what  their  capabili¬ 
ties  are  as  to  tarqet/threat  detection. 

j)  Development  of  target  identification  techniques. 
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Use/Validity: 


k)  Utilization  of  all  significant  system  capabilities. 

l)  Development  of  hostile  target  nullification 
techniques. 

m)  Completion  of  the  scenario  until  the  threats  are 
destroyed  or  the  system  capabilities  are  depleted  or 
successfully  countered. 

The  scenario  should  identify  which  tactics  appear  to  be 
feasible,  which  may  overstress  the  system,  and  which  mission 
functions  must  be  broken  down  to  lower,  more  detailed  levels 
in  order  to  determine  their  feasibility  and  operation  within 
the  context  of  the  scenario.  If  possible,  the  user  command 
should  be  contacted  to  obtain  information  to  assist  the  de¬ 
velopment  of  the  scenarios. 

All  of  these  data  wil  be  used  while  performing  the  various 
analysis  techniques  such  as  functional  flows,  decision/action 
diagrams,  and/or  act ion/ informat ion  requirements.  Table 
3.9-1  indicates  the  output  applications,  including  mission 
effectiveness  criteria,  which  result  from  performing  mission 
scenarios. 


Mission  scenarios  are  compared  to  other  analysis  techniques 
in  Table  3.9-4.  Review  of  the  table  Indicates  that  the  sce¬ 
narios  should  be  developed  early  in  the  program.  They  are 
simple  to  perform  because  no  elaborate  symbology  or  function¬ 
al  links  are  required.  They  are  used  for  gross  analysis  more 
than  for  detailed  analysis.  Tasks  may  be  described  for 
single  operator  systems  or  pultioperator  systems.  Mission 
scenarios  may  take  a  long  tlnv  to  develop  if  the  proposed 
system  Is  relatively  new  and  uniiue.  Additional  time  may  be 
required  to  detail  new  data  fo<  subsystem  capabilities.  The 
scenarios  wili  be  used  by  managers  as  well  as  analysts. 

Their  relative  cost  and  cost  effectiveness  may  be  rated  as 
average. 
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3. 9. 4. 3  Functional  Flow  Diagrams 

Description:  Functional  flow  diagrams  are  the  most  popular  systems 

technique  used  for  the  determination  of  system 
requirements.  Starting  with  system  or  mission  objectives, 
functional  flows  are  developed  iteratively  for  more  and 
more  detailed  system  requirements  down  to  the  level  of  spe¬ 
cific  operator  tasks.  Functional  flow  diagrams  can  provide 
a  detailed  outline  of  all  system  requirements.  They  may  be 
used  as  an  extensive  checklist  of  system  functions  that 
must  be  considered  in  assuring  the  ability  of  the  system  to 
perform  the  mission.  This  analysis  of  system  functions  is 
required  to  determine  solutions  for  later  trade  studies. 
Functional  flows  are  necessary  to  determine  effectively 
which  system  functional  elements  should  be  performed  by 
operators,  equiDment,  software  or  some  combination  of 
these.  In  general,  during  the  construction  of  higher  level 
flows,  no  distinction  should  be  made  between  operator, 
equipment,  or  software  implementation  of  system  functions. 
The  lack  of  distinction  is  for  the  purpose  of  conducting 
unbiased  system  trade  studies. 

Functional  flow  diagrams  are  often  referred  to  as  function¬ 
al  block  diagrams,  functional  flows,  or  functional  flow 
block  diagrams.  All  of  these  terms  refer  to  the  same  anal¬ 
ysis  technique.  It  has  undoubtedly  evolved  from  the  use  of 
schematic  block  diagrams  that  depict  the  relationships  be¬ 
tween  various  equipment  items  in  a  system.  The  major  dif¬ 
ference  between  the  schematic  diagram  and  the  functional 
flow  is  the  addition  of  the  verb  to  the  noun  label  in  each 
schematic  block.  By  the  use  of  verb-noun  functions,  the 
system  is  prevented  from  becoming  prematurely  committed  to 
an  arbitrary  design  implementation  solution.  A  function 
may  be  defined  as  a  verb-noun  phrase  that  must  be  accom¬ 
plished  by  a  system.  All  functions  can  be  broken  down  or 
divided  into  more  detailed  functions. 
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Procedure: 


Sample  functional  flows  are  shown  in  Figure  3.9-2,  These 
flows  are  constructed  by  arranging  In  system  sequential  order 
all  of  the  various  functions  that  are  believed  to  pertain  to 
a  particular  system  (or  subsystem,  depending  on  level  of 
detail).  Each  function  Is  a  verb-noun  combination.  Occa¬ 
sionally  nouns  are  assumed  and  adjectives  are  added.  Each 
Individual  function  ;s  contained  within  a  rectangular  block. 
Each  block  is  number*. -1  for  reference  more  or  less  according 
to  its  sequence  on  the  page.  These  numbers  remain  with  the 
function  as  long  as  the  function  is  unique. 

If  the  function  is  repeated  in  other  portions  of  the  total 
series  of  functional  flows,  the  same  number  should  be  used 
and  the  block  may  be  drawn  as  a  reference  block.  Each  func¬ 
tional  flow  diagram  contains  a  reference  to  its  next  higher 
functional  flow  through  the  use  of  a  reference  block.  Refer¬ 
ence  blocks  may  also  be  used  to  indicate  functions  occurring 
at  the  same  level  on  different  pages.  The  blocks  in  Figure 
3.9-2  that  are  broken  in  the  middle  are  reference  blocks. 

The  numbers  are  important  to  insure  traceability  either  back 
to  the  higher  level  functions  or  between  functions. 

The  functional  flow  symbology  used  in  Figure  3.9-2  is 
typical.  The  direction  between  the  function  blocks  indicates 
the  normal  sequence  of  occurrence  of  system  functions. 
Contrary  to  the  ground  rules  for  constructing  schematics,  the 
arrows  between  functional  flow  blocks  should  show  the  general 
flow  of  the  diagram  toward  the  right  and,  if  necessary, 
down.  Arrows  should  not  be  used  on  either  the  top  or  bottom 
of  the  blocks.  They  should  enter  the  block  from  the  left  and 
exit  to  the  right. 
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FIGURE  3 .9-2 :  SAMPLE  FUNCTIONAL  FLOW  DIAGRAMS 
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Wherever  arrows  are  joined  or  split  out,  they  should  be 
connected  by  an  "and",  "or",  or  "and/or"  gates  or  junctions 
as  Indicated  In  the  sample.  The  significance  of  the  "and" 
junction  is  that  all  of  the  following  or  preceding  func¬ 
tions  must  be  performed.  The  "or"  junction  indicates  a 
choice  between  two  or  more  of  the  following  or  preceding 
functions  as  to  which  one  is  performed.  The  "and"  and  “or" 
junctions  may  be  combined  if  It  will  not  cause  confusion 
and  page  space  is  limited. 

In  addition  to  the  previous  discussion,  the  point  should  be 
made  that  a  function  is  that  which  must  be  accomplished  by 
the  system  and  that  all  functions  can  be  broken  down  or 
divided  into  more  detailed  functions.  Top  level  and  first 
level  functions  tend  to  be  identical  for  similar  systems 
(e.g.,  perform:  preflight,  taxi,  takeoff,  etc.).  A  spe¬ 
cific  operational  requirement  may  call  for  modification  to 
these  higher  level  functions;  however,  the  changes 
generally  occur  to  the  lower  level  functions.  For  large 
programs,  such  as  a  complete  air  vehicle  system,  they  are 
gross  system  operations.  The  second  level  functions  would 
then  tend  to  describe  system  operational  (or  maintenance) 
functions  within  the  various  mission  phases.  The  third 
level  may  define  specific  functions  with  measurable  perfor¬ 
mance  units.  Functional  allocation  between  operators, 
equipment  and  software  may  occur  at  this  level.  Fourth 
level  functions  may  be  the  level  at  which  gross  operator 
task  analysis  may  occur.  The  total  concept  of  functional 
level  detail  or  definition  must  be  based  on  the  total  size 
or  scope  of  the  particular  system  to  be  analyzed. 

Naturally,  the  smaller  the  system  being  worked,  the  more 
detailed  the  corresponding  numerical  level  of  functional 
analysis  will  be.  larger  systems  or  programs  will  require 
more  levels  to  get  to  the  same  degree  of  detail. 
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In  view  of  this  possible  ambiguity  as  to  functional  level 
definition  versus  program  scope,  it  is  recommended  that  all 
parties  concerned  (e.g.,  customer  and  contractor)  agree  on 
the  definitions  before  considerable  effort  is  expended  on 
this  or  similar  techniques.  The  definition  of  functional 
levels  is  not  as  important  as  the  assurance  that  analysis  is 
conducted  to  a  sufficient  degree  of  detail  to  determine  sig¬ 
nificant  operator  performance  requirements,  particularly  the 
details  of  critical  operator  tasks.  The  reference  number 
groups  recommended  for  use  with  each  of  the  levels  is  as 
follows:  1.0,  2.0,  3.0,  etc.,  for  top  level  functions:  1.1, 

1.2,  1.3,  etc.,  for  first  level  functions:  1.1.1,  1.1.2, 
2.1.1,  etc.,  for  second  level  functions;  and  1.1. 1.1, 

1.1. 1.2,  2. 1.1.1,  etc.,  for  third  level  functions  and  so  on. 

Once  the  functional  flows  are  constructed,  the  functions  and 
subfunctions  should  be  reviewed  and  analyzed  in  depth  for 
probable  variations  related  to  the  system  requirements.  Even 
during  early  development,  both  alternative  mission  require¬ 
ments  and  the  expected  downstream  developmental  impact  of 
such  alternatives  should  be  appraised  to  produce  an  early 
estimate  of  likely  crew  interface  requirements,  capability, 
special  provisions  needed,  potential  problems  and  probable 
solutions. 

In  come  cases,  the  analyst  may  also  need  to  produce 
preliminary  workload  data  and  to  provide  information  for  man¬ 
ning  and  training  estimates.  In  any  case,  he  must  anticipate 
a  wide  variety  of  possible  requirements  to  form  a  judgment 
for  both  crew  performance  feasibility,  support  requirements 
and  development  needs. 
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Some  of  the  essential  features  to  remember  about  the 
procedure  for  constructing  functional  flows  are  as  follows: 

a)  Functional  flow  blocks  must  contain  a  verb  and  a 
noun. 

b)  It  is  essential  to  initiate  the  flows  on  a  system 
framework  and  without  any  allocation  to  operator, 
equipment,  or  software. 

c)  Each  expanded  level  of  functional  flow  will  con¬ 
tain  mora  and  more  detailed  information.  The  de¬ 
tail  may  be  carried  on  to  as  many  levels  as 
appropriate.  It  is  normally  necessary  to  go  to  at 
least  the  third  level. 

d)  Functions  are  numbered  in  a  manner  which  preserves 
continuity  of  function  and  logical  breakout  from 
function  origin. 

e)  The  diagram  should  be  organized  so  that  one  can 
easily  find  the  input  and  follow  the  flow  through 
the  function  blocks  to  the  resulting  output. 

f)  It  is  generally  good  practice  to  limit  the  size  of 
the  diagrams.  They  should  be  divided  up  if  too 
large  for  foldout  pages  in  documents.  Reference 
blocks  may  be  used.  If  designed  for  display  on 
walls,  the  functional  flows  may  be  of  relatively 
large  size. 

Use/Validity:  Functional  flow  diagrams  are  extremely  useful  to  the  human 
factors  engineer  for  a  number  of  reasons.  The  functional 
block  numbering  system  provides  a  rationalized  traceability 
from  lower  to  higher  level  functions  and  between  functions 
at  the  same  level.  Functional  flows  are  flexible  in  that  a 
change  in  one  part  of  a  total  functional  flow  generally 
causes  minimal  effect  on  other  parts.  Because  of  this, 
they  are  easy  to  use  to  show  the  effects  of  preliminary 
functional  allocation  trades  to  man,  machine  or  software. 
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Because  of  this  flexibility  and  ease  of  use,  they  are  an 
Ideal  technique  to  use  for  the  rapid  analysis  of  system 
functions  proposed  by  other  program  personnel  such  as  sub¬ 
system  designers.  Functional  flows  are  the  ideal  way  to 
show  the  relationships  between  functions.  They  may  be  con¬ 
structed  in  such  a  manner  as  to  show  as  many  as  forty  or 
fifty  different  functions  on  one  foldout  page.  If  wall 
space  is  available,  complete  systems  or  subsystems  may  be 
laid  out,  depending  on  the  level  of  detail  desired.  Func¬ 
tional  flows  are  relatively  easy  to  develop.  Whereas  some 
human  factors  engineering  analysis  techniques  require 
special  training  prior  to  their  use,  the  functional  flow 
diagram  requires  only  minimal  training.  The  functional 
flow  diagrams  are  also  a  relatively  fast  analysis  technique 
and  accordingly,  they  tend  to  be  very  cost  effective.  The 
only  reason  for  not  using  this  analysis  technique  would  be 
to  use  another  technique  in  its  place,  such  as  the 
decision/action  diagram,  (discussed  in  next  paragraph, 

3. 9. 4. 4),  which  incorporates  most  of  the  same  features  of 
the  functional  flow.  Functional  flows  do  not  contain  in¬ 
formation  pertaining  to  decisions  and  time-based  informa¬ 
tion  flow,  although  functional  flows  tend  to  be  time 
sequential.  Functional  flows  generally  do  not  indicate 
operator,  equipment,  or  software  allocations,  except  at  a 
lower,  more  detailed  level. 

The  data  for  the  functional  flows  oriqinally  come  from  the 
mission  profiles  and  scenarios  that  are  developed  during 
the  operations  analysis  program  effort.  Data  for  more  de¬ 
tailed  lower  level  functional  flows  also  come  directly  from 
the  higher  level  flow  diagrams  and  from  the  subsystem  de¬ 
sign  groups.  In  a  similar  manner  to  all  other  analysis 
techniques,  the  functional  flow  diagrams  are  not  an  end  in 
themselves.  There  is  little  or  no  point  in  constructing 
them  if  they  are  to  be  completed  only  to  be  filed  away. 
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As  more  and  more  detailed  functional  flows  are  developed, 
specific  system  requirements  begin  to  emerge.  These  re¬ 
quirements  may  then  be  documented  by  incorporation  into 
system  specifications.  As  previously  indicated,  functional 
flows  are  used  to  assist  in  the  performance  of  functional 
trades  (i.e.,  trades  performed  to  choose  between  or  among 
two  or  more  functional  alternatives).  The  results  of  the 
trades  should  evolve  into  detailed  system  requirements  or 
specifications.  The  functional  flows  are  seldom  adequate 
to  develop  detailed  system  requirements  where  operators  are 
involved.  Additional  analysis  techniques  such  as  time 
lines,  requirements  allocation  sheets,  and/or  operational 
sequence  diagrams  need  to  be  generated  to  develop  system 
requirements  pertaining  to  system  decision  functions  or 
time  constraints. 

Review  of  Table  3.9-1  indicates  several  specific  output  ap¬ 
plications  that  result  from  performing  functional  flow 
analysis.  Table  3.9-4  indicates  numerous  evaluation  char¬ 
acteristics  of  the  functional  flow  as  compared  to  other 
analysis  techniques.  The  technique  is  best  used  during 
concept  formulation  and  after  DSARC  I  phases  of  the 
program.  It  is  relatively  simple  to  perform  this  technique 
at  a  gross  function  level.  As  more  detail  is  required, 
other  techniques  should  be  selected.  It  is  best  suited  to 
gross  analysis.  Analysis  of  several  simultaneous  functions 
is  no  problem  to  perform  with  functional  flows.  The  time 
to  perform  functional  flows  is  relatively  short;  but  that 
is,  of  course,  a  function  of  the  total  analysis  effort 
involved.  Functional  flows  may  be  expected  to  be  used  by 
both  managers  and  analysts.  Their  relative  cost  to  perform 
is  from  low  to  medium  and  their  relative  cost  effectiveness 
is  from  medium  to  high. 
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In  summary,  functional  flows  provide  a  detailed  and  compre¬ 
hensive  inventory  of  all  system  requirements  and  an  exten¬ 
sive  checklist  of  system  functions  and  factors  that  must  be 
considered  in  assuring  ability  to  perform  the  mission. 
Properly  structured,  the  inventory  will  proceed  from  func¬ 
tional  indentures  common  to  all  similar  systems  (e.g., 
aircraft),  through  indentures  peculiar  tn  an  aircraft  type 
(e.g.,  fighters)  and  on  to  functional  elements  that  are 
specific  to  mission  operations.  Detailed  analysis  of  the 
functions  is  required  to  determine  basic  methods  of 
achievement,  possible  equipments,  and  man/equipment  trades 
in  order  to  effectively  determine  which  elements  should  be 
performed  by  equipment  and  which  should  be  performed  by 
man . 

3. 9. 4. 4  Oecis ion/Action  Diagrams 

Description:  The  decision/action  diagram  is  a  technique  similar  to  func¬ 

tional  flows.  It  is  used  to  show  the  flow  of  required  sys¬ 
tem  data,  in  terms  of  operations  and  decisions.  Like  func¬ 
tional  flow  block  diagrams,  decision/action  diagrams  may  be 
developed  and  used  at  various  levels  of  detail.  The 
}  initial  decision/action  diagram  charts  are  concerned  with 

*  gross  functions  without  regard  to  whether  functions  are 

performed  by  man,  machine,  software,  or  some  combination  of 
these.  The  decision/action  diagrams  prepared  subsequent  to 
tentative  man-machine-software  function  allocations  will 
reflect  this  allocation  in  the  decisions,  operations,  and 
branching  which  are  represented.  At  the  program  concept 
formulation  stage,  however,  these  charts  would  ordinarily 
be  prepared  at  a  detailed  level  only  for  the  more  critical 
1  man-machine  functions. 
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This  technique  may  also  be  referred  to  as  information  flow 
charts,  decision  logic  diagrams,  or  operation/decision 
diagrams.  The  term,  information  flow  charts,  generally 
refers  to  a  type  of  decision/action  diagram  that  has  a 
vertical  orientation  on  the  page  rather  than  the  left  to 
right  horizontal  orientation  that  decision/action  diagrams 
use.  Special  symbology  may  also  be  used  with  the  informa¬ 
tion  flow  charts  at  a  more  detailed  level  to  indicate 
allocations  to  man  or  machine  (e.g.,  single  line  symbols 
mean  manual,  double  line  mean  automatic). 

The  decision/action  diagrams  are  so  similar  to  functional 
flow  diagrams  that  the  use  of  both  techniques  is  not 
recommended.  The  most  significant  difference  between  the 
two  techniques  is  the  addition  of  the  decision  blocks 
(diamonds)  to  the  functional  flow  diagrams.  The  decision/ 
action  diagrams  are  generally  used  when  the  program  is 
software  oriented. 

In  that  it  records  the  sequence  of  operations  and  decisions 
which  must  be  performed  to  satisfy  a  definite  system 
function,  the  decision/action  diagram  is  similar  to  the 
flow  charts  used  by  computer  programmers.  Both  charts  are 
based  on  binary  choice  decisions  and  intervening 
operations.  There  are  two  important  reasons  for  using 
binary  decision  logic  as  a  standard  in  performing 
decision/action  diagramming: 

a)  To  expedite  communications  through  use  of  simple 
yet  universally  appicable  conventions. 

b)  To  provide  for  easy  translation  of  decision/action 
flow  charts  into  logic  flow  charts  for 
computerized  sections  of  the  system. 
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A  decision  at  a  general  level  may  split  into  several 
decisions  at  a  more  detailed  level,  for  example: 

General  level:  -  Do  any  targets  need 

identification  processing? 

More  specific  level:  -  Do  any  newly  entered  targets 

need  identification  processing? 

-  Do  any  target  tracks  need  con¬ 
firmation  of  tentative 
identification? 

-  Do  any  confirmed  identifications 
need,  rechecking? 

Each  of  these  more  detailed  decisions  may  have  associated 
with  it  one  or  more  detailed  operations.  Similarly,  an  op¬ 
eration  at  a  general  level  may  break  down  into  more  de¬ 
tailed  decisions  and  operations. 

The  example  in  Figure  3.9-3  is  a  gross  level  detection  and 
tracking  function.  No  functional  allocation  has  been  made 
to  man  or  machine.  Note  that  at  this  level  the  chart  is 
apolicable  to  several  detection  and  tracking  systems  -  the 
decisions  and  operations  are  essentially  common  between 
them.  Even  here,  however,  the  usefulness  of  the  flow  chart 
diagramming  technique  is  apparent  because  it  makes  the 
analyst  begin  to  consider  implementation  alternatives,  such 
as: 

a)  By  what  means  can  any  given  signal  be  compared 
with  known  targets  in  the  system? 

b)  How  can  probable  targets  be  marked  so  their  reap¬ 
pearance  can  be  readily  recognized? 
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Tne  Information  necessary  for  the  initiation  of  decision/ 
logic  diagrams  comes  from  the  mission  profiles  and 
scenarios.  Data  for  more  detailed  lower  level  decision/ 
logic  diagrams  may  come  directly  from  higher  level  flow  di¬ 
agrams  and  from  sybsystem  design  groups  as  equipment  de¬ 
tailed  characteristics  become  well  defined. 

Procedure:  The  procedure  for  constructing  decision/action  diagrams  is 

essentially  the  seme  as  that  for  functional  flow  diagrams. 
They  are  constructed  by  arranging  in  sequential  order  all 
of  the  functions  and  decisions  that  pertain  to  a  system  or 
i  subsystem  (depending  on  level  of  detail).  Each  function  is 
a  verb-noun  combination  with  occasional  adjectives  or  other 
modifiers.  Each  function  phrase  is  relatively  short  and  is 
contained  within  a  rectangular  block.  Each  decision  func¬ 
tion  is  placed  in  a  diamond  shaped  outline  symbol  and  is 
written  in  a  question  format  that  may  be  answered  with  a 
binary,  yes-no,  response.  Roth  the  functional  action 
blocks  and  the  decision  diamonds  should  be  given  reference 
numbers  in  a  manner  similar  to  the  numbers  assigned  to 
functional  flow  diagrams. 

The  numbers  are  important  to  ensure  traceability  between 
decision/action  blocks.  The  decision  diamond  blocks  may  be 
drawn  in  solid  or  dasned  lines  to  indicate  primary  decision 
functions  or  shared  decision  functions,  respectively.  The 
use  of  arrows  between  function/decision  blocks  is  similar 
to  functional  flows.  Note  that  flow  paths  should  be 
complete.  Every  path  should  either  recirculate  or  end  in  a 
valid  exit  with  a  reference  block.  The  junction  between 
arrows  are  handled  with  "and",  "or",  or  "and/or"  gates  in 
the  same  manner  as  with  functional  flows.  (Reference  para¬ 
graph  on  Functional  Flow  Diagrams,  Procedures). 
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Figure  3-9*3:  Sample  Dec isfon/Actlon  Diagram 


Use/Val idity: 


The  results  of  the  dec  is  ion/act  ion  diagram  analysis  are 
used  to  develop  specific  system  requirements  and  assist  in 
the  performance  of  trade  studies.  Additional  analysis 
techniques  such  as  time  lines  are  almost  always  needed  fol¬ 
lowing  the  construction  of  the  decision/action  diagrams  in 
order  to  investigate  the  effect  of  the  critical  system 
parameter,  time.  Worthwhile  computer  simulations  have  been 
successfully  performed  with  the  addition  of  time  data  to 
detailed  decision/action  diagrams  that  include  preliminary 
allocations  of  functions  to  operators.  Table  3.9-1  indi¬ 
cates  several  specific  output  applications  that  result  from 
performing  decision/action  diagrams.  The  technique  is  well 
suited  to  initial  development  of  software  programs  in 
general,  and  display  software  in  particular. 

Review  of  Table  3.9-4  indicates  a  preference  for  performing 
decision/action  diagrams  during  the  earliest  phase  of  a 
program.  They  are  considered  to  be  either  average  or 
simpler  than  average  in  complexity,  but  they  must  still  be 
considered  slightly  more  complex  than  functional  flows 
because  of  the  added  decision  functions.  They  are  Detter 
used  for  gross  analysis  and  may  be  used  to  analyze  several 
simultaneous  functions.  They  require  a  relatively  short  to 
medium  time  to  perform  and  cost  an  average  (or  less)  amount 
of  manpower  effort.  They  rate  higher  than  average  in  cost 
effectiveness.  The  decision/action  diagrams  are  useful  to 
both  analyst  for  the  determination  of  detailed  system 
requirements,  and  to  HFE  managers  for  the  determination  of 
more  general  program  or  system  requirements. 
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3. 9. 4. 5  Act  Ion/ Informat  Ion  Requirements 

Description:  Given  the  functional  flows,  or  decision/action  diagrams, 

analytic  procedures  for  performing  preliminary  functional 
allocation  are  somewhat  dependent  on  the  analyst  and  his 
objectives.  For  the  purpose  of  performing  functional  allo¬ 
cation  trades,  one  alternative  technique  is  to  make  the  al¬ 
location  from  the  level  of  the  detail  provided  in  the  func¬ 
tional  flows.  However,  experience  suggests  that  more  de¬ 
tail  than  that  provided  at  the  functional  level  may  ta  de¬ 
sirable  before  making  allocation  trades.  A  format  which 
has  been  useful  in  producing  this  detail  in  an  appropriate 
context  is  the  system  “action/information  requirements". 
Figure  3.9-4  illustrates  such  a  form.  Use  of  this  format 
helps  in  defining  those  specific  actions  necessary  to  per¬ 
form  a  function  and,  in  turn,  those  specific  information 
elements  that  must  be  provided  to  perform  the  action.  It 
breaks  up  the  referenced  "functional  requirement"  into  use¬ 
ful  groupings  of  "action  requirements"  and  "information 
requirements".  This  particular  sample  format  is  expanded 
to  include  detailed  aspects  of  the  function  such  as  related 
information  requirements,  sources,  and  problems.  Related 
accident  features  and  survey  commentary  are  also  included 
in  this  example.  However,  the  precise  format  of  this  par¬ 
ticular  form  does  not  need  to  be  rigidly  controlled. 

Procedure:  The  procedure  for  developing  or  completing  action/infor¬ 

mation  requirements  forms  is  much  more  informal  than  that 
for  most  analysis  techniques.  Often  the  three  columns  il¬ 
lustrated  on  the  left  side  of  the  form  illustrated  in 
Figure  3.9-4  are  all  that  arc  used.  The  first  column  is 
used  to  list  the  function  and  function  number  from  the 
functional  flow  diagrams:  The  second  column  is  used  to 
list  each  of  the  action  requirements  indicated  by  the 
function.  The  third  column  is  used  to  list  the  information 
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Figure  3.9-4:  Sample  Action/Information  Requirements  Form 


requirements  that  come  from  the  listed  function.  If  more 
detail  is  desired  for  the  preparation  of  the  allocation 
trades,  additional  columns  may  be  added  on  the  right  side 
of  the  form.  In  the  example  in  Figure  3.9-4,  related  in¬ 
formation  requirements,  sources,  and  problems  are  listed. 

A  second  column  lists  related  accident  features  and  the 
third  column  lists  any  other  commentary.  In  this  case,  the 
column  is  used  for  survey  results  pertinent  to  the  function 
being  scrutinized.  Additional  data  could  be  listed,  such 
as  the  capabilities  of  operators  or  equipment  for  handling 
these  functional  requirements. 

Use/Validity:  Table  3.9-4  compares  the  use  of  this  analysis  technique  to 
other  techniques.  The  action/information  requirements 
forms  should  be  used  after  the  functional  flows  but  before 
the  functional  allocation  trades.  The  appropriate  time 
during  the  program  to  perform  this  analysis  technique  would 
therefore  be  during  the  concept  formulation  or  after  DSARC 
I.  The  technique  is  of  average  complexity.  It  is  used  at 
analysis  levels  sufficiently  detailed  to  perform  functional 
allocations.  It  is  used  to  analyze  one  function  at  a  time. 
It  requires  an  average  amount  of  time  to  perform  and  is  of 
much  more  use  to  analysts  than  to  managers.  Its  relative 
cost  and  cost  effectiveness  to  perform  are  average.  It  is 
not  recommended  if  there  is  relatively  little  difficulty  in 
obtaining  sufficiently  detailed  functions  from  which  func¬ 
tional  allocation  trades  may  be  performed. 

Use  of  this  particular  technique  provides  the  analyst  with 
the  information  to  exercise  several  options:  a)  he  can 
identify  equipment  which  satisfies  the  system  requirements, 
b)  he  can  perform  associated  man/equipment  capability 
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trades  for  preliminary  functions  allocation,  c)  he  can  inte¬ 
grate  similar  or  correlated  system/action/information  re¬ 
quirements  to  develop  new  concepts,  or  d)  he  can  easily  pair 
action  requirements  with  possible  control  hardware  and  infor¬ 
mation  requirements  with  possible  display  hardware. 

The  information  used  to  construct  these  forms  comes  primarily 
from  the  functional  flows.  Additional  data  may  be  obtained 
from  subsystem  design  engineers.  The  results  obtainable  from 
this  analysis  technique  are  used  by  human  factors  engineers 
in  the  performance  of  functional  allocation  trades. 

3. 9. 4. 6  Function  Allocation  Trades 

Description:  With  the  completion  of  the  functional  flow  diagrams, 

decision/action  diagrams,  and/or  action/information 
requirements,  it  is  appropriate  to  perform  preliminary 
trade-off  studies  of  man-machine  allocations  for  each  of  the 
functions  being  considered.  Too  often  the  allocations  are 
based  only  on  past  experience,  or  worse  yet,  the  allocations 
are  simply  arbitrary.  A  rationalized  choice  of  functions  is 
necessary  for  optimum  system  design. 

These  man-machine  allocations  provide  the  baseline  for 
down-stream  efforts  relating  to  crew  task  definition, 
control/display  operations  requirements,  crew  station  config¬ 
uration  concepts,  workload  evaluation  and  crew  station 
design,  development  and  evaluation.  Additionally,  function 
allocations  dictate  crew  workload  and  significantly  affect 
manning,  training  and  procedures  requirements.  Early 
appraisals  of  the  allocation  impact  on  these  requirements  are 
necessary  as  part  of  the  initial  human  engineering  review 
process.  Early  appraisals  that  anticipate  program  and  opera¬ 
tional  requirements  are  reflected  in  the  earliest  system  de¬ 
velopment  phases. 


forking  in  conjunction  with  project  subsystem  designers 
(perhaps  as  a  team  to  do  this  task)  and  using  the  function¬ 
al  flows,  etc.,  plus  their  past  experience  with  similar 
systems,  the  human  factors  engineer  makes  a  preliminary  al¬ 
location  of  the  actions,  decisions,  and/or  functions  shown 
in  the  previously  used  charts  and  diagrams  to  operators, 
equipment,  or  software.  The  assignment  of  the  functions, 
actions,  and/or  decisions  to  operators,  equipment,  or  soft¬ 
ware  must  be  based  on:  a)  the  known  capabilities  and 
limitations  of  operators,  b)  the  state-of-the-art  perfor¬ 
mance  of  hardware  and  software,  and  c)  estimated  perfor¬ 
mance  to  be  required  in  terms  of  speed,  accuracy,  and 
load.  The  need  for  a  cooperative  effort  between  subsystem 
designers  and  human  factors  engineers  at  this  point  is 
extremely  important.  Each  must  contribute  to  make  the 
allocations  meaningful. 

There  are  three  specific  techniques  recommended  to  perform 
the  details  of  the  function  allocation  trade.  The  first 
technique  is  simply  that  of  "trial  and  error"  substitution 
of  each  of  the  alternatives  into  a  system  or  subsystem 
model.  Each  alternative  is  then  evaluated  on  a  basis  of 
total  system  or  subsystem  reliability  or  speed.  This 
technique  has  some  obvious  drawbacks.  It  is  not  recommend¬ 
ed  for  a  systems  analysis  where  a  large  number  of  functions 
need  to  be  allocated.  The  technique  lends  itself  for  use 
to  computer  analysis  much  better  than  manual  (paper  and 
pencil)  analysis.  Computer-aided  techniques  that  may  be 
used  for  this  type  of  analysis  are  described  in  following 
paragraphs  of  this  guide. 

The  second  technique  is  oased  on  an  evaluation  matrix 
(Figure  3.9-5).  Candidate  subsystem  functions  are  listed 
and  compared  against  the  "Fitts  List"  (Ref.  5,  AF$C  OH  1-3) 
man-machine  capabilities  (see  Table  3.9-6).  The  form  used 
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Figure  3.9-5:  Sample  Functional  Allocation  Screening  Worksheet  (Evaluation  Matrix) 


Table  3.9-6:  Man/Machine  Capabilities  Fitts  List 


HAN  EXCELS  IN 


Detection  of  certain  forme  of  very 
low  energy  levels 

Sensitivity  to  so  extremely  wide 
verlety  of  stimuli 

Perceiving  patterns  end  asking 
general lzations  about  them 

Detecting  signals  in  high  noise 

levels 

Ability  to  store  large  amounts  of 
information  for  long  periods  - 
and  recalling  relevant  facts  at 
appropriate  moments 

Ability  to  exercise  judgment 
where  events  cannot  be  completely 
defined 

Improvising  and  adopting  flexible 
procedures 


Ability  to  react  to  unexpected 
low-probability  events 

Applying  originality  in  solving 
problems:  l.e.,  alternative 
solutions 

Ability  to  profit  from  experi- 
lerice  and  alter  course  of  sctlon 

Ability  to  perform  fine  manipula¬ 
tion,  expeclally  where  misalignment 
appears  unexpectedly 

Ability  to  continue  to  perform 
when  overloaded 


Ability  to  reason  inductively 


MACHINES  EXCEL  IN 


Monitoring  (both  men  and  machines) 


Performing  routine,  repetitive,  or 
very  precise  operations 

Responding  very  quickly  to  control 
signals 

Exerting  great  force,  smoothly  and 
with  precision 

Storing  and  recalling  large  amounts 
of  information  in  short  time- 
periods 


Performing  complex  and  rapid 
computation  with  high  accuracy 


Sensitivity  to  stimuli  beyond  the 
range  of  human  sensitivity  (Infra¬ 
red,  radio  waves,  etc.) 

Doing  many  different  things  at 
one  time 

Deductive  processes 


Insensitivity  to  extraneous  factors 


Ability  to  repeat  operations  very 
rapidly,  continuously,  and  pre¬ 
cisely  the  same  way  over  a  long 
period 

Operating  in  environments  which  are 
hostile  to  man  or  beyond  human 
tolerance 


Reference  AFSC  DH  1-3 
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Procedure: 


to  perform  this  technique  is  called  a  functional  allocation 
screening  worksheet.  Plausible  operator  roles  or  equipment 
functions  (e.g.,  operating,  monitoring,  maintaining, 
programming,  communicating,  etc.)  are  identified  using  the 
screening  worksheet.  By  comparing  the  functions  to  be  per¬ 
formed  with  the  inherent  capabilities  of  man  or  machine  to 
accomplish  the  functions,  operator  and  equipment  tasks  are 
allocated.  The  comparison  is  evaluated  and,  based  on  the 
analyst's  judgment,  a  weighted  numerical  score  is  assigned 
to  each  function/capabilities  criteria  relationship. 

The  third  technique  is  also  based  on  an  evaluation  matrix 
and  is  often  referred  to  as  a  design  evaluation  matrix.  In 
this  technique,  candidate  subsystem  functions  are  listed 
and  compared  against  selected  criteria  for  allocation 
(response  time,  error  rate,  operability,  cost,  etc.).  As 
In  the  case  of  the  screening  worksheets,  the  evaluation 
criteria  are  weighted  since  some  factors  are  obviously  more 
important  than  others.  Each  of  the  function/evaluation 
criteria  relationships  is  assigned  a  numerical  score,  as  to 
how  each  function  best  meets  the  selected  evaluation 
criteria.  This  third  technique  is  well  suited  for  use  in 
complying  with  MIl-H-46855  requirements  (i.e.,  Paragraph 
3. 2. 1.4  of  that  specification).  Human  engineering  criteria 
such  as  that  In  Mll-STD-1472  may  be  used  as  the  selection 
evaluation  criteria. 

The  procedure  for  accomplishing  the  first  of  the  three 
functional  allocation  trade  techniques  is  actually  the  same 
as  the  procedures  for  accomplishing  some  of  the  other  human 
factors  analysis  techniques.  In  other  words,  once  one  of 
the  alternatives  for  a  particular  function  is  tentatively 
chosen,  the  alternative  should  be  evaluated  for  use  by 
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performing  one  of  the  analysis  techniques  on  it.  For 
example,  the  time  line  analysis  technique  should  be  used  to 
evaluate  an  allocation  trade  where  either  operators  or 
equipment  are  chosen  to  perform  time  critical  tasks.  The 
resulting  allocation  choice  is  then  the  solution  that  best 
meets  the  system  time  requirements.  In  a  similar  manner, 
other  allocation  trades  may  be  accomplished  to  evaluate 
man-machine  functional  performance  in  terms  of  reliabil¬ 
ity.  The  following  paragraphs  will  indicate  which  tech¬ 
niques  are  best  suited  for  testing  particular  performance 
parameters. 

Functional  allocation  screening  worksheets  are  constructed 
by  listing  each  of  the  several  functions  to  be  allocated  on 
the  left  side  of  the  worksheet.  Two  sets  of  evaluation 
criteria  are  listed  across  the  top  of  the  sheet.  The  first 
set  pertains  to  operator  capabilities;  the  second  set 
pertains  to  equipment  capabilities.  Each  of  the  capabili¬ 
ties  evaluation  criteria  is  taken  from  the  often  used 
"Fitts  List"  (Table  3.9-6).  In  order  to  balance  out  each 
of  the  evaluation  capabilities,  each  one  against  all  the 
others,  numerical  weightings  have  been  assigned  as  appro¬ 
priate  for  the  system  being  analyzed.  For  example, 
"response  to  signals"  may  be  particularly  important  as 
compared  to  "inductive  reasoning"  and  it  should  therefore 
be  weighted  more  heavily.  Although  not  a  part  of  the 
"Fitts  List",  such  factors  as  cost  may  be  added  to  these 
other  characteristics.  Such  parameters  are  generally  con¬ 
sidered  for  evaluation  using  the  design  evaluation  matrix 
technique  discussed  In  the  following  paragraph.  Whenever 
an  evaluation  characteristic  (across  the  top  of  the  sheet) 
is  applicable  to  a  listed  function  (left  side  of  sheet)  a 
weighted  "X"  is  placed  in  the  column/row  intersection. 
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The  actual  evaluation  is  made  by  totaling  up  each  of  the 
weighted  "X'o"  for  the  "operator”  versus  the  "equipment" 
allocation.  The  results  of  the  allocation  are  tabulated  in 
the  far  right-hand  columns  as  either  "operator",  "both",  or 
"equipment".  The  "both"  column  is  used  when  the  sums  from 
both  sides  of  the  worksheet  come  nut  to  be  within  approxi¬ 
mately  80%  of  each  other.  In  this  case,  a  more  detailed 
analysis  may  be  required  to  obtain  a  detailed  breakout  of 
operator  or  equipment  allocation.  If  a  more  precise  evalu¬ 
ation  o*  each  of  the  functions  is  desired,  a  numerical 
score  (e.g.,  1-5)  may  be  used  to  indicate  how  well  a  par¬ 
ticular  "Fitts  List"  evaluation  characteristic  applies  to  a 
function.  This  procedure  is  used  in  the  Figure  3.9-5 
construction.  The  number  entered  in  the  column/row 
intersection  is  the  weighted  evaluation  factor  times  the 
score.  As  with  the  simpler  method  indicated  above,  the 
total  scores  are  added  up  on  each  side  of  the  worksheet  to 
obtain  a  proposed  functional  allocation.  It  should  be 
noted  that  whereas  this  technique  does  not  insure  the 
absolutely  best  allocation  of  functions,  it  goes  a  long  way 
beyond  the  "gutfeel"  method  so  often  used. 

Construction  of  the  design  evaluation  matrix  is  similar  to 
the  functional  evaluation  screening  worksheet  in  that  the 
functions  are  listed  along  the  left  side  and  the  evaluation 
factors  are  listed  across  the  top  of  the  sheet.  The  main 
difference  is  that  the  trade  to  be  performed  is  not  neces¬ 
sarily  between  man  or  machine  for  a  particular  single  func¬ 
tional  listing.  The  trade  to  be  performed  Is  between  each 
of  the  functional  alternatives  listed  along  the  left  side 
of  the  sheet.  Another  difference  between  the  two  tech¬ 
niques  is  that  the  functional  lists  for  the  design  evalua¬ 
tion  matrix  tend  to  be  of  several  equipment  alternatives 
rather  than  just  operator  versus  equipment  alternatives 
(See  Figure  3.9-5). 
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The  evaluation  characteristics  listed  across  the  top  of  the 
sheet  pertain  more  to  performance  parameters  thar  to  inherent 
capabilities.  The  evaluation  characteristics  should  be 
weighted  and  the  suitability  of  a  particular  functional  al¬ 
ternative  to  an  evaluation  characteristic  should  be  scored  on 
a  scale  of  1  to  5.  The  addition  of  each  of  the  weighted 
scores  determines  the  best  alternative. 

Use/Validity:  Initial  function  allocations  are  typically  obtained  from  in¬ 
formation  taken  from  mission  requirements,  functional  flows, 
or  other  preliminary  analysis  diagrams.  Function  aspects 
such  as  difficulty,  priority  and  criticality  are  appraised 
and  operator/eguipment  methods  for  meeting  the  requirements 
are  evaluated.  The  results  of  the  function  allocation  trade 
are  used  to:  a)  determine  impact  of  crew  tasks,  skills  and 
information  needs;  b)  appraise  related  crew  task  capability 
and  limitations;  c)  identify  corresponding  control /display 
concepts;  d)  trade  specific  and  detailed  control /display/ 
crew  performance  capabilities;  e)  perform  extensive  task 
analysis  and  workload  evaluations;  and  f)  identify 
control/display/crew  operations  requirements  in  order  to 
proceed  to  g)  crewstation  configuration  development. 

These  techniques  are  compared  to  other  human  factors  engi¬ 
neering  analysis  techniques  in  Table  3.9-4.  Functional  allo¬ 
cation  studies  are  best  performed  early  In  the  program.  Al¬ 
though  there  are  variations  in  the  choice  of  specific 
techniques,  they  all  may  be  considered  to  be  of  average 
complexity.  They  may  be  used  for  either  gross  or  detailed 
analysis  of  functions  but  are  used  more  often  for  gross  func¬ 
tional  allocation. 


Several  functions  may  be  simultaneous  by  the  use  of  one 
techrijue  worksheet.  The  time  taken  to  perform  the  analy¬ 
sis  should  be  short  to  medium,  depending  on  the  scope  of 
the  functional  allocation  effort.  The  results  of  the  ef¬ 
fort  will  be  used  equally  by  managers  and  analysts.  The 
relative  cost  and  cost  effectiveness  are  both  average. 

3. 9. 4. 7  Time  Lines 

Description:  Time  lines  (or  timelines)  are  one  or  the  most  basic  tech¬ 

niques  used  by  HFE  analysts.  The  two  parameters  in  which 
HFE  analysts  are  most  interested  are  time  and  errors. 

There  is  no  better  way  to  analyze  just  the  parameter  of  op¬ 
erator  time  performance  than  by  the  use  of  time  lines. 

Time  lines  serve  two  purposes.  First,  they  permit  an 
appraisal  of  time-critical  sequences  to  verify  that  all 
necessary  events  can  be  performed.  Secondly,  they  provide 
an  integrated  task  time  chart  to  assess  the  occurrence  of 
incompatible  tasks  and  to  serve  as  a  baseline  for  workload 
evaluation.  A  typical  time  line  example  is  shown  in  Figure 
3.9-6. 

Procedure:  Each  time  line  should  be  related  to  a  higher  level  func¬ 

tional  requirement .  The  functional  flow  title  and  number 
should  be  indicated  on  the  time  line  sheet  for  reference 
(see  Figure  3.9-6  sample).  Other  information  such  as 
location  of  the  function  and  the  type  of  function  is 
desirabl?.  Each  of  the  subfunctions  or  tasks  are  numbered 
and  listed  along  the  left  side  of  the  sheet.  The  time 
units  uf  interest  (hours,  minutes,  or  seconds)  are  indi¬ 
cated  and,  at  the  same  time,  a  scale  of  suitable  length 
selected  such  that  the  total  time  period  of  interest  fits 
onto  the  worksheet.  It  is  recommended  that  once  the  scale 
for  a  sheet  is  chosen,  it  be  adhered  to  for  all  portions  of 
that  time  line  sheet. 
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Use/Validity:  Almost  all  the  techniques  previously  presented  are  sources 
of  data  to  be  used  in  preparing  time  lines.  Generally,  the 
most  common  source  of  material  for  a  time  line  analysis  is 
a  detailed  level  functional  flow  diagram;  one  that  is  suf¬ 
ficiently  detailed  to  have  tasks  allocated  to  the  operators 
as  the  result  of  functional  allocation  trades.  Table  3.9-1 
shows  the  wide  variety  of  applications  or  outputs  'or  which 
tir,.„  line  analysis  data  may  be  used. 

Table  3.9-4  indicates  the  relationship  between  time  lines 
and  the  numerous  technique  evaluation  characteristics.  Re¬ 
view  of  this  table  indicates  that  time  lines  are  best  used 
during  concept  formulation  and  after  DSARC  I  but  before 
DSARC  II.  They  are  of  average  complexity  to  develop,  and 
they  are  equally  useful  for  analysis  of  either  gross  or  de¬ 
tailed  operator  procedures.  They  are  well  suited  for  the 
analysis  of  either  an  individual  operator's  tasks  or  sever¬ 
al  operators'  tasks,  as  long  as  all  tasks  are  placed  on  the 
time  line  sheet.  Compared  to  other  analysis  techniques, 
time  lines  take  slightly  less  than  an  average  amount  of 
time  to  perform.  They  are  easy  to  read  and  understand,  and 
they  are  therefore  of  use  to  both  managers  and  analysts. 
Their  relative  cost  is  medium  and  their  cost  effectiveness 
is  slightly  above  average.  Although  not  indicated  in  Table 
3.9-5,  they  are  extremely  cost  effective  for  use  in 
analyzing  simple  operator  tasks  where  time  is  the  critical 
factor. 


3. 9. 4. 8  Flow  Process  Charts 


Description: 


Procedure: 


Flow  process  charts  (FPC’s)  are  basically  plots  of  the 
sequence  of  operator  activities  or  Information  transfer  as 
a  part  of  a  system.  The  plots  or  flow  of  activities  and 
information  exchange  are  plotted  in  time  sequence.  Figure 
3.9-7  Is  an  example  of  such  a  plot.  It  is  very  similar  to 
the  information  flow  chart  mentioned  previously.  The  dif¬ 
ference  between  the  two  techniques  is  that  the  FPC's  use  a 
wider  variety  of  symbology  and  are  generally  performed  at  a 
more  detailed  operator  task  level.  The  FPC  symbology  is 
shown  in  Figure  3.9-8.  The  symbology  is  adopted  from  the 
ASME  (American  Society  of  Mechanical  Engineers),  flow  chart 
standards. 

The  FPC  is  oriented  vertically,  frequently  with  a  time 
scale  to  one  side  cr  another  of  the  function  or  task 
symbology.  Each  task  performed  by  the  operator  is  recorded 
with  the  proper  symbology  (see  Figure  3.9-7)  and  with  a 
brief  description  of  the  task.  A  time  value,  and  perhaps  a 
distance,  are  also  recorded  if  appropriate.  Start  and  stop 
points  of  the  charted  activity  are  indicated. 

In  preparing  these  charts,  the  HFE  analyst  should  ensure 
that  all  logical  possibilities  are  included,  all  loops  are 
completed  or  terminated  in  a  valid  exit,  and  all  tasks  are 
capable  of  being  performed  by  the  operator.  The  following 
aspects  must  be  considered:  a)  how  each  operator  will  make 
decisions,  b)  what  the  criteria  are  to  be  used  for  decision 
making  and  c)  what  information  requirements  must  be  met  to 
provide  a  basis  for  decision  making. 
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ANY  TARGET  TRACKS  IN  SYSTEM? 


PRESS  SEQ  BUTTON 

PUT  NEXT  TARGET  IN  TRACK  LIST  UNDER  CLOSE  CONTROL 

ADVANCE  HOOK  ON  CRT  TO  COORDINATES  FOR  TRACK  UNDER  CLOSE  CONTROL 

IS  TARGET  VIDEO  PRESENT? 

DOES  HOOK  LINE  UP  WITH  PRESENT  TARGET  POSITION? 

ENABLE  TRACK  BALL  AND  REPOSITION  IT  TO  MOVE  HOOK  OVER  TARGET 
PRESS  POB.  CORR.  BUTTON 

ADD  LATEST  POSITION  DATA  TOGETHER  WITH  TIME  TO  MEMORY.  COMPUTE 
AND  STORE  COURSE  AND  SPEED.  PERIODICALLY  UPDATE  TARGET  POSITION 

ANY  TARGET  FAIL  TO  BE  UPDATED  WITHIN  CRITICAL  TIME? 

DISPLAY  "RECOMMENDED  DROP  TRACK"  ALERT 
DROP  ALERTED  TRACK? 

HOOK  AND  PRESS  DROP  TRACK  BUTTON 

DELETE  TRACK  FROM  MEMORY 

Q  HUMAN  OPERATION  Q  MACHINE  OPERATION 
<^>  HUMAN  DECISION  MACHINE  DECISION 


Figure  3.9-7:  Sample  Flow  Process  Chart 


Symbology 


Operate 


Inspect 


Transmit*  - 


an  action  function,  to  accomplish  or  continue 
a  process.  (Soaetiaes  used  for  received 

information) 

to  aonltor  ot  verify  quantity  or  quality.  An 
inspection  occurs  when  an  object  is  examined. 
(Sometimes  used  for  action) 

to  pass  Information  without  changing  its  form. 


Receipt* 


to  receive  information  in  the  transmitted  form. 
(Sometimes  used  for  stored  information) 


Decision 


to  evaluate  and  select  a  course  of  action  or 
inaction  based  on  receipt  of  information. 


* 


Storage 


Mode  of  transmission  and  rece 


to  retain.  (Sometimes  used  for  transmitted 
information) 

lpt  is  Indicated  by  a  code  letter  within  th»i 


and 


symbols. 


V  ••  Visual 

E  -  Electrical/Electronic 

S  -  Sound  (verbal) 

IC  -  Internal  Communication 

EX  -  Externtl  Communication 

T  -  Touch 

M  -  Mechanically 

W  -  Walking 

H  -  Hand  Deliver 


(Special  combinations  or  symbols  are  shown  in  Figure  3.9-10) 


Figure  3.9-8:  F PC  and  OSD  Syirbology 
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Use/Validity:  The  purpose  of  constructing  the  flow  process  charts  is  to 
aid  in  developing  and  evaluating  concepts  for  each  operator 
station.  If  a  single  operator  station  is  being  analyzed, 
it  is  a  good  technique  to  use;  however,  if  more  than  one 
station  is  being  analyzed,  a  separate  chart  must  be  devel¬ 
oped  for  each  station.  The  operational  sequence  diagram 
(OSO),  which  is  discussed  in  the  following  paragraph,  is  a 
better  technique  to  use  for  multiple  operator  station 
analysis. 

Table  3.9-1  indicates  the  applications  or  outputs  from  the 
FPC's.  A  comparison  of  the  FPC  technique  with  all  of  che 
other  manual  techniques  is  indicated  in  the  Table  3.9-4. 

In  summary,  the  FPC  should  be  used  during  the  earlier  pro¬ 
gram  phases.  It  is  of  average  complexity  and  may  be  used 
for  analysis  of  detailed  tasks.  The  relative  time  to  per¬ 
form  the  FPC's  is  average  as  compared  to  other  manual  anal¬ 
ysis  techniques.  FPC's  are  used  by  analysts  more  than 
managers.  Their  relative  cost  to  perform  is  average,  as  is 
their  relative  cost  effectiveness. 

3. 9. 4. 9  Operational  Sequence  Oiagrams 

Description:  The  operational  sequence  diagram  (OSD)  is  probably  the  most 

powerful  single  manual  analysis  technique  that  the  HFE 
analyst  can  use.  This  is  because  of  all  the  outputs  and 
applications  that  derive  from  its  use  (Ref.  Table  3.9-1). 

It  is  particularly  useful  for  the  analysis  of  highly  com¬ 
plex  systems  requiring  many  time  critical  information- 
decision-action  functions  between  several  operators  and 
equipment  items. 


115 


Procedure: 


The  0S0  has  been  used  on  numerous  Navy  programs  such  as 
Polaris,  ASMS,  VPX,  and  the  Air  Force  E-3A  (AWACS).  It  was 
derived  from  the  flow  process  charts  (FPC).  It  retains  the 
same  basic  attributes  of  the  FPC.  It  is  a  graphic  presen¬ 
tation  of  operator  tasks  as  they  relate  sequentially  to 
both  equipment  and  other  operators.  OSD  symbology  is  also 
adapted  from  the  ASME  flow  chart  standards.  The  0S0  is  an 
FPC  expanded  In  terms  of  channels  or  work  stations. 

8y  using  symbology  to  indicate  actions,  inspections,  data 
transmitted  or  received,  data  storage,  or  decisions,  the 
OSD  shows  the  flow  of  information  through  a  system.  The 
information  flow  is  shown  in  relation  to  both  time  and 
space  (work  stations).  The  OSD  may  be  used  to  develop  and 
present  the  system  reaction  to  specified  inputs.  It  is  one 
of  the  cheapest  and  quickest  ways  to  simulate  the  system. 
Whereas  mockups  and  prototypes  may  be  more  complete  for 
some  simulation  aspects,  they  are  more  expensive.  Computer 
programs  are  also  generally  more  expensive  depending  upon 
how  often  they  are  used.  In  the  OSD,  the  interrelation¬ 
ships  of  operators  and  equipment  (man-machine  interfaces) 
are  easily  visualized.  Whenever  information  transferred  is 
mismatched  with  the  format  to  be  received,  interface  prob¬ 
lems  are  clearly  indicated.  Operator  activities  are  se¬ 
quentially  categorized.  Decision  and  action  functions  are 
clearly  identified  and  task  frequency  and  load  become 
obvious. 

A  sample  OSD  is  shown  in  Figure  3.9-9.  An  explanation  of 
0S0  symbology  is  included  in  Figures  3.9-8  and  3.9-10.  In 
a  similar  manner  to  the  FPC's,  the  flow  of  events  and  tasks 
is  always  from  the  top  of  the  sheet  toward  the  bottom.  The 
operators  and  machines  are  entered  into  the  column  headings 
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on  the  OSD.  It  generally  proves  convenient  to  place  in 
adjacent  columns  the  operators  and  the  machines  with  which 
they  interface.  Also,  it  helps  to  group  together  all  of 
the  operators  and  equipment  of  a  specific  functional  divi¬ 
sion  (e.g.,  Weapons  Control).  In  some  cases,  the  operators 
or  maintainers  and  equipment  in  a  system  will  have  been 
specified  by  the  time  the  OSD  is  constructed.  However,  if 
the  men  and  machines  have  not  been  specified,  the  analysts 
will  have  to  specify  them  tentatively.  In  either  case,  in 
the  process  of  doing  the  OSD,  it  may  be  found  that  too  many 
or  too  few  operators  and/or  machines  have  been  selected. 

The  reason  for  doing  the  analysis  is  to  "drive  out"  crew 
size  and  interface  requirements. 

The  OSD  is  initiated  by  the  first  event  designated  by  the 
scenario  (Reference  previous  paragraph).  The  event  and 
event  times  are  written  in  the  twc  left-hand  columns.  All 
of  the  machines  or  men  who  will  receive  the  input  are  shown 
and  the  transmission/reception  mode  is  noted  by  using  the 
appropriate  letter  code.  The  subsequent  actions  taken  by 
the  crew/equipment  (operations,  transmissions,  etc.)  as 
they  react  to  the  input  are  shown.  External  outputs  are 
plotted  in  the  far  right-hand  column.  As  the  reactions  are 
plotted,  the  analyst  should  be  cognizant  of  the  time  re¬ 
quired  to  perform  the  actions.  The  process  of  plotting 
the  inputs  and  subsequent  reactions  is  continued  as  dic¬ 
tated  by  the  events  given  in  the  scenario  or  narrative.  No 
attempt  is  made  to  keep  the  actual  space  between  scenario 
time  events  proportional  to  the  time  itself. 

It  is  important  to  remember  that  the  reader  of  an  OSD 
should  be  clearly  shown  the  operation  of  the  system,  and 
all  of  the  ster  -  shown  on  the  J~,D  should  be  described  by  a 
brief  notation  describing  the  process  or  action.  As  with 
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Figure  3.9-9.  Semple  Operational  Sequence  Diagram 
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Exchange  of  intonation  or  diacues- 
lon  by  two  principals  involved. 

Uaed  with  appropriate  aource  codes. 


Acknowledgement  of  receipt  of  Information 
uaed  with  appropriate  aource  codes. 


Continuous  flow  of  Information  throughout  event 
Receipts  are  picked  off  where  needed  in  sequent 
without  repeating  entire  process.  Time  inter¬ 
vals  may  be  indicated  as  shown. 


Double  symbols  indicate  automatic  transmission, 
receipt,  storage  or  operation. 


DECISION 

LEFT  NO 
RIGHT  YES 


INSPECTION 

NO  GO 
GO 


A  repeated  process  uqually  repeated  until  a 
desired  condition  exists  before  continuing. 
Note:  The  last  repeat  of  several  may  be  shown 
In  normal  sequential  order  to  give  a  clearer 
picture  of  che  event. 


Figure  3.9-10:  Special  Combinations  of  OSD  Symbols 
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the  case  of  the  FPC,  the  HFE  analyst  should  be  sure  that 
all  logical  possibilities  are  included,  all  loops  are  com¬ 
pleted  or  terminated  In  a  valid  exit,  and  all  tasks  are  ca¬ 
pable  of  being  performed  by  the  operators. 

Use/Validity:  The  reason  the  0S0  Is  so  useful  in  terms  of  outputs  is 
simply  that  so  much  must  go  into  It.  The  Integration  of 
all  the  data  that  go  into  a  typical  OSD  is  generally  a 
tedious  and  time  consuming  process.  Experience  has  shown 
that  the  construction  of  OSD's  requires  trained  individuals 
with  analytic  skills. 

The  information  to  construct  an  0S0  may  come  from 
scenarios,  functional  flow  diagrams,  time  lines, 
decision/action  diagrams,  work  station  layouts,  or  other 
sources.  If  the  HFE  analyst  is  dependent  on  other  organi¬ 
zations  for  this  information,  h?  must  conduct  numerous 
interviews  of  other  organization  personnel  or  have  an 
extremely  efficient  program  requirements  documentation  ef¬ 
fort  to  draw  on. 

Table  3.9-1  indicates  several  specific  output  applications 
that  result  from  performing  an  OSD  analysis.  Table  3.9-4 
indicates  the  numerous  evaluation  characteristics  of  the 
OSD  as  compared  to  other  analysis  techniques  and  indicates 
the  OSD  should  be  used  during  the  earlier  program  phases. 

It  Is  a  complex  technique  and  may  be  used  for  analysis  of 
detailed  tasks.  It  is  particularly  useful  for  the  analysis 
of  several  tasks  that  are  occurring  almost  simultaneously 
between  several  operators  or  between  several  operators  and 
equipment.  Because  of  the  complexity  of  the  OSD,  it  tends 
to  take  a  relatively  long  time  to  perform.  Its  cost  to 
perform  is  relatively  high  (two  man-years  for  the  ASMS  con¬ 
cept  formulation  phases),  but  its  payoff  in  terms  of  a 
paper  system  test  and  verification  gives  it  an  "average" 
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relative  cost  effective  rating.  Also,  it  should  be  empha¬ 
sized  that  the  OSD  Is  like  any  other  paper  simulation 
technique  In  that  It  must  be  validated  as  soon  as  practical 
In  an  environment  closely  similar  to  the  actual  working 
environment.  Although  much  more  complex,  OSD's  are 
somewhat  similar  to  decision/action  diagrams.  Often  when 
decision/action  diagrams  are  used,  OSD's  are  not. 

Another  technique  that  is  similar  to  the  OSD  is  the  func¬ 
tional  sequence  diagram  (FSD).  Its  format  is  very  nearly 
identical  to  the  OSD's.  It  is  easier  to  construct  but  does 
not  provide  as  much  useful  information  as  the  OSD.  The 
difference  between  the  two  techniques  is  that  the  FSD  does 
not  make  a  distinction  between  operators  and  equipment. 

3.9.4.10  Task  Descriptions 

Description:  Task  descriptions,  as  a  distinct  analysis  technique,  are 

not  used  as  much  today  as  they  were  several  years  ago. 

Newer  manual  and  computer-aided  techniques  are  being  used 
in  place  of  them.  However,  they  are  presented  here  because 
they  still  have  unique  characteristics  that  are  suited  to 
particular  analysis  applications.  Task  descriptions  are 
one  additional  human  factors  engineering  tool  that  can  be 
used  to  help  define  personnel  requirements  in  complex 
systems.  Taking  the  data  developed  by  the  use  of  previous 
analysis  techniques,  task  descriptions  can  be  developed 
which  will: 

a)  Test  the  man/machine  system  interface  to  ensure 
compatibilities  with  operator  abilities; 

b)  Contribute  to  the  development  of  training 
programs,  training  manuals,  and  job  aids  for 
personnel  who  will  be  involved  in  the  operation 
and  maintenance  of  the  system;  and 

r.)  Assist  in  the  personnel  procurement  and  associated 
manpower  planning  process. 
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Procedure: 


Task  descriptions  are  developed  from  the  functional  alloca¬ 
tion  process  data.  Task  descriptions  provide  a  basic  re¬ 
ference  for  subsequent  design  and  development  of  the  entire 
personnel  subsystem.  A  task  description  is  essentially  a 
statement  of  basic  task  requirements.  It  can  assist  in  de¬ 
sign  finalization  by  identifying  operability  or  maintain¬ 
ability  problem  areas,  or  by  defining  operator  activities 
with  specific  equipment.  Task  descriptions  received  con¬ 
siderable  emphasis  in  the  Air  Force  Systems  Command  Manual 
375-5  (Ref.  33)  system  engineering  process  several  years 
ago.  In  a  few  instances,  the  same  worksheet  forms  are 
still  being  used  today. 

The  level  of  detail  in  an  adequate  task  description  depends 
largely  upon  the  complexity  and  criticality  of  a  given 
system,  and/or  the  expected  levels  of  difficulty  in  train¬ 
ing  and  manning  the  system.  Generally,  the  level  of  detail 
for  specifying  task  activities  is  about  the  same  as  that 
used  in  an  instruction  manual  for  a  novice.  A  good  task 
description  could  easily  become  a  procedural  manual  for  the 
job.  Figure  3.9-11  is  an  example  of  a  detailed  task 
description,  and  it  illustrates  the  kinds  of  elements  that 
must  be  identified. 

Task  descriptions  should  proceed  from  general  task 
statements  to  specific  display,  control,  decision  activity 
details.  In  the  example  of  Figure  3.9-11,  functions  that 
have  been  allocated  to  man  during  the  functional  allocation 
process  are  listed  along  the  left  side  of  the  analysis 
form.  Under  the  heading  "Elements"  the  task  activities  are 
listed.  These  are  tasks  that  may  be  classified  as  actions, 
perceptual  motor  activities,  straight  monitoring, 
communicating  and  decision  making  or  problem  solving. 
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POSITION:  PILOT 
OUTY:  VEIIIC 


Figure  3.9-11:  Sample  Task  Description 


The  associated  controls  and/or  displays  are  listed  along 
with  the  activity.  Remarks  that  have  to  do  with  the 
activity  are  included  in  the  far  right-hand  column.  These 
remarks,  which  might  include  contingencies  which  can 
severely  affect  the  mission  or  system  success,  are 
identified;  particularly  because  of  their  impact  on  opera¬ 
tor  skill  level  requirements.  Major  environmental 
conditions  affecting  a  mission  cycle,  or  any  segment  of  it 
should  be  included  in  the  remarks  column.  Machine 
malfunctions  that  might  occur  during  a  critical  mission 
task  should  also  be  included.  If  there  is  a  particularly 
high  probability  of  human  error,  this  data  should  be  indi¬ 
cated  in  the  remarks  column.  The  corresponding  times  for 
each  of  the  operator  task  elements  have  been  estimated  and 
included  in  a  column  next  to  the  task  column.  It  should  be 
noted  that  task  descriptions  need  not  be  highly  structured, 
but  can  be  modified  to  fit  the  requirements  of  various 
systems. 

Use/Validity:  Table  3.9-4  summarizes  the  characteristics  of  task 
descriptions  as  compared  to  all  the  other  analysis 
techniques.  Task  descriptions  are  prepared  at  any  time 
during  the  program;  however,  they  are  of  less  value  during 
the  time  period  following  OSARC  III.  They  are  relatively 
simple  to  construct  and  are  used  for  either  gross  or  de¬ 
tailed  analysis.  Task  descriptions  are  used  to  describe 
several  simultaneous  tasks  but  are  better  used  to  show  the 
single  thread  sequential  relationship  of  one  task  occur¬ 
rence  at  a  time.  The  time  required  to  prepare  a  task  de¬ 
scription  is  average  as  compared  to  any  other  analysis 
technique.  The  table  indicates  that  both  managers  and 
analysts  have  equal  use  of  the  technique.  The  relative 
cost  to  prepare  a  complete  task  description  Is  average. 

The  relative  cost  effectiveness  is  average.  The  technique, 
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being  more  narrative  in  form  than  pictorial,  gives  less 
visibility  to  items  of  analysis  interest  such  as  task  or 
time  relationships.  Problems  which  ere  generally 
iiscovered  as  a  result  of  performing  time  line  analysis  are 
not  as  apparent  as  a  result  of  using  this  technique.  The 
length  of  the  time  blocks  used  in  time  line  sheets 
"displays"  the  time  relation  between  each  block.  This  re¬ 
lationship  is  harder  to  see  as  just  a  number  in  task 
descriptions. 

3.9.4.11  Workload  Analysis 

Description:  Workload  analyses  provides  an  appraisal  or  the  extent  of 

crew  task  loading,  based  on  the  sequential  accumulation  of 
task  times.  Application  of  this  technique  permits  an  eval¬ 
uation  of  the  capability  of  the  crew  to  perform  all  assign¬ 
ed  tasks  in  the  time  allotted  by  mission  constraints.  As 
capability  is  confirmed,  hardware  design  requirements  can 
be  more  precisely  designated.  Conversely,  as  limitations 
are  exposed,  alternate  function  allocations  or  crew  task 
assignments  are  considered  and  implemented. 

Workload  analysis  or  workload  profiles,  as  they  are  often 
referred  to,  are  a  graphic  presentation  of  an  operator’s 
workload  constructed  by  plotting  percentage  of  task 
involvement  against  a  time  base  (sea  Figure  3.9-12).  Al¬ 
though  workload  analysis  depicts  individual  activity,  its 
greatest  effectiveness  is  realized  when  several  operator/ 
maintainer  positions  are  plotted  together  on  the  same 
graph.  By  doing  this,  any  unbalanced  workload 
distributions  among  the  operators  become  readily  apparent. 
Earliest  possible  workload  appraisals  are  needed  to  assure 
that  resulting  task  loads  are  within  the  scope  of  the  crew 
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Figure  3.9  12.  Semple  Workload  Analysis  Profile 


size  and  capability.  Workload  analysis  was  developed  to 
verify  that  no  combination  of  tasks  required  more  task  load 
capacity,  or  time  to  perform  than  is  available.  One  oV  the 
more  recent  concepts  in  workload  analysis  has  been  to 
divide  the  operator  tasks  into  cateqories  corresponding  to 
his  perceptual -motor  channels.  This  analysis  refinement 
does  not  necessarily  have  to  be  accomplished  in  order  to 
successfully  perform  workload  analysis.  However,  the  more 
detailed  the  analysis  the  better  the  output  data.  In  some 
situations,  operators  can  effectively  perform  more  than  one 
task  at  one  time.  However,  it  is  obvious  that  an  operator 
cannot  accomplish  two  tasks  simultaneously  if  both  tasks 
require  the  use  of  a  single  perceptual -motor  channel  nearly 
100%  of  the  time.  The  workload  analysis  chart  exposes  such 
conditions  when  properly  developed. 

When  such  conditions  are  noticed,  it  is  apparent  that  one 
of  two  things  must  be  done.  Either  a  task  must  be  given  to 
another  operator  or  the  operator  must  be  provided  with  some 
type  of  equipment  assistance. 

The  task  loading  estimates  may  come  from  several  sources. 
For  example,  the  task  may  be  the  same  as,  or  similar  to, 
another  task  in  another  system  which  is  in  actual 
operation.  Task  time  data  from  previous  systems  is 
generally  the  most  reliable  since  it  has'  been  verified  in 
practice.  When  such  information  is  not  available,  the  next 
best  data  is  from  operators  who  have  performed  similar 
tasks.  It  Is  desirable  to  get  estimates  from  several  oper¬ 
ators  since  their  evaluations  will  vary.  The  HFF.  analyst 
must  provide  the  operator  with  enough  detail  to  enable  him 
to  make  an  estimate. 
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Procedure: 


When  experienced  operators  or  other  data  sources  are  not 
available,  the  HFE  analyst,  together  with  knowledgeable 
project  designers,  must  make  an  "educated  guess"  about  the 
task  workload  Implications.  The  HFE  analyst  will  have  to 
do  what  he  does  with  all  Droblems  of  this  sort;  he  will 
have  to  break  the  task  down  into  Its  simplest  elements  and 
extrapolate  from  what  he  knows  about  other  subtask 
elements. 

In  application,  workloads  are  estimated  at  either  a  gross 
level  or  detailed  level  in  terms  of  both  time  and  number  of 
perceptual-motor  channels  considered  for  analysis.  As 
workload  situations  tend  to  become  more  critical,  shorter 
time  increments  are  examined.  Also,  as  workload  increases 
for  a  given  situation  and  as  the  situation  becomes  more 
critical,  it  is  desirable  to  make  workload  assessments  on 
the  basis  of  each  of  th^  operator's  perceptual -motor 
channels.  These  are  generally  listed  as:  external  vision 
(distance  vision),  internal  vision  (within  the  cockpit  or 
console  panel  area),  left  hand,  right  hand,  feet, 
cognition,  audition,  and  verbal  channels. 

Workload  calculations  are  based  on  estimates  of  the  time 
required  to  perform  a  given  task  divided  by  the  time 
allowed  or  available  to  perform  the  task.  The  analyst  is 
cautioned  that  if  he  evaluates  workload  by  considering  each 
of  the  distinct  perceptual-motor  channels  he  cannot  equate 
a  75%  loadinq  on  each  channel  to  an  overall  75%  loading. 

The  precise  summation  effects  of  all  or  several  of  the 
channels  cannot  be  accurately  predicted.  Ouite  possibly 
the  results  of  a  75%  loading  on  each  channel  would  result 
in  a  total  overload  situation  (>100%).  The  analyst  is 
also  cautioned  not  to  average  workload  over  the  time 
increments  being  considered.  A  workload  estimate  of  100% 
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and  an  estimate  of  50%  for  two  sequential  tasks  occurring 
within  a  given  time  increment  must  be  considered  as  an 
overall  estimate  of  100%  (not  75%).  If  It  is  necessary  to 
provide  visibility  to  the  50%  loading  situation,  then  the 
time  increments  must  be  broken  down  into  smaller  time 
periods.  The  point  of  the  analysis  is  to  discover  signifi¬ 
cant  workload  conditions  including  peaks,  not  to  mask  them 
out. 

In  general,  workloads  over  100%  are  inacceptable,  between 
75%  and  100%  are  undesirable,  and  under  75%  are  acceptable 
provided  that  the  operator  is  given  sufficient  work  to 
remain  reasonably  busy.  Prior  to  its  current  revisions, 
MlL-H-46855  contained  an  appendix  that  described  the 
conditions  where  operator  workload  analysis  should  be 
performed.  The  implication  was  that  operator  loading  in 
excess  of  75%  should  receive  special  scrutiny. 

Since  the  process  of  estimating  workload  is  based  on  the 
estimate  of  time  required  to  do  the  task,  it  is  only  as 
accurate  as  that  data.  It  is  also  limited  by  the  knowledge 
of  the  time  available  to  do  the  task,  and  it  is  limited  by 
the  unknown  discrete  channel  summation  effects.  Depending 
on  these  variables  alone,  the  accuracy  of  most  workload 
assessments  are  probable  in  the  ±20%  range.  If  more 
accurate  assessments  are  required,  full  scale  simulations 
of  the  crew  tasks  may  oe  necessary. 


Use/ Validity: 


< 


The  workload  analysis  may  be  made  up  of  a  simple  continuous 
chart  from  the  beginning  to  end  of  a  mission,  or  there  may 
be  several  charts,  each  of  which  expands  a  particularly 
critical  segment  of  the  mission.  As  previously  indicated, 
the  time  scale  should  be  commensurate  with  task  complexity, 
i.e.,  15  minute  intervals  may  be  all  that  is  necessary  for 
simple  workload  analysis  evaluations  and  5  second  intervals 
may  be  required  for  more  complex  tasks.  Whatever  intervals 
are  used  should  be  common  for  the  total  group  of  tasks  and 
operators  when  they  interact. 

Table  3.9-1  indicates  the  applications  or  outputs  of  work¬ 
load  analysis.  An  evaluation  of  workload  analysis  as 
compared  to  other  techniques  is  shown  in  Table  3.9-4. 
Workload  analysis  is  most  generally  performed  after  DSARC  I 
when  sufficient  other  analysis  has  been  performed  in  order 
to  develop  the  input  data  to  workload  analysis.  It  may 
continue  past  OSARC  II  and  possibly  past  DSARC  III.  The 
complexity  of  this  analysis  is  average  as  compared  with 
other  techniques.  It  may  be  used  to  perform  a  gross  or  top 
level  (several  minutes  at  a  time)  analysis  of  operator 
workload  or  a  very  detailed  (a  few  seconds  at  a  time) 
analysis.  If  several  workload  profiles  are  combined  on  one 
page,  it  may  be  used  to  compare  several  simultaneous 
tasks.  The  time  to  perform  this  manual  workload  assessment 
is  about  average  as  compared  to  other  analysis  techniques. 
8ecause  of  the  definition  of  work  overload  and  the  notion 
of  the  use  of  separate  perceptual-motor  channels,  this 
technique  is  best  used  by  analysts  alone.  If  used  by 
managers,  a  detailed  explanation  must  accompany  the  data. 
The  relative  cost  to  perform  the  analysis  is  average,  as  is 
the  relative  cost  effectiveness  as  compared  with  other 
analysis  techniques. 
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3.9.4.12  Correlation  Matrix 

Description:  Tne  correlation  matrix,  or  chart,  is  one  of  the  simplest 

and  easiest  analysis  techniques  to  use.  It  is  constructed 
in  a  manner  similar  to  a  highway  map  mileage  chart.  It  is 
generally  used  after  the  development  of  OSD's  for  the  pur¬ 
pose  of  summing  up  all  of  the  links  between  each  of  the 
iperators,  operator  workstations,  and/or  equipment.  Figure 
3.9-13  is  an  example  of  a  correlation  matrix.  It  is  a 
summary  of  the  communications  occurring  during  a 
hypothetical  function.  Although  correlation  matrices  are 
of  use  by  themselves  to  determine  the  frequency  of  use  of 
the  various  links  or  interfaces  between  system  man/machine 
components,  they  are  more  often  used  as  an  intermediate 
analysis  step  between  the  OSD  and  link  analysis.  The  fol¬ 
lowing  section  indicates  how  the  correlation  matrix  data 
are  used  as  an  input  to  link  analysis.  The  reason  for 
having  a  list  of  the  relative  frequencies  of  use  of  the 
communcation  paths,  or  whatever  sort  of  man/machine  links 
there  are,  is  t  •  locate  each  of  the  man/machine 
workstations  (or  function)  so  that  the  paths  between  them 
are  as  short  as  practicable.  cor  example,  if  crewman  "A" 
is  required  to  pass  ten  times  as  many  messages  to  crewman 
"6"  as  he  does  to  crewman  "C",  then  it  stands  to  reason 
that  he  shou’d  be  located  much  closer  to  crewman  "6” . 

Procedure:  All  of  the  man/machine  components  of  the  system  that  are 

listed  across  the  top  of  the  OSD  and  that  are  of  interest 
to  the  analyst  are  listed  in  a  vertical  column.  As  can  be 
seen  from  the  example  in  Figure  3.9-13,  parallel  lines  are 
extended  to  the  right  at  angles  up  and  down  from  each  of 
the  listed  workstations.  This  results  in  diamond  shaped 
blocks  at  the  intersections  of  the  rows  coming  out  from 
each  listed  workstation.  The  number  of  links  between  each 
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CREW  POSITION 


I  of  the  listed  man/machine  workstations  are  counted  up  from 

{  the  0S0  (each  link  should  be  drawn  in  on  the  OSD).  The 

total  quantity  of  links  is  placed  in  the  diamond  shaped 
|  block  that  represents  the  intersection  of  the  rows  coming 

!  out  from  the  workstations. 

| 

|  Although  not  absolutely  required,  it  may  be  just  as  impor¬ 

tant  to  add  a  letter  symbol  as  an  indication  of  the 
estimated  criticality  of  the  data  transfer,  or  links,  be- 
!  tween  workstations.  The  intersecting  blocks  and  total 

:  matrix  would,  of  course,  have  to  be  made  large  enough  to 

put  all  of  the  data  as  to  number  of  links  of  each  kind 
(high,  medium,  low  criticality)  in  each  of  the  intersecting 
blocks.  Letter  symbology  may  also  be  used  to  indicate  the 
type  of  data  link,  e.g.,  direct  voice,  interphone,  TTY, 

Use/Validity:  Table  3.9-1  shows  the  various  applications  of  the  correla¬ 
tion  matrix  data.  Table  3.9-4  evaluates  the  technique 
against  all  the  other  analysis  techniques.  As  previously 
indicated,  the  timing  for  the  performance  of  the  correla¬ 
tion  matrix  is  dependent  on  the  OSD.  It  should  be  perform¬ 
ed  during  the  Concept  Formulation  phase  or  after  D3ARC  I  or 
whenever  the  OSD  analysis  has  taken  place.  The  correlation 
matrix  is  a  very  simple  technique  to  use.  It  is  best  used 
to  summarize  man/machine  links  at  a  detailed  level  of 
analysis.  Of  course,  it  is  used  to  summarize  these  links 
or  data  paths  for  several  tasks  for  several  workstations 
occurring  over  a  period  of  time  that  was  analyzed  by  the 
OSD.  Because  of  its  simplicity  the  correlation  matrix 
takes  only  a  very  short  time  to  perform.  Correlation 
matrices  are  useful  to  both  managers  and  analysts.  The 
relative  cost,  to  perform  is  low  and  the  cost  effectiveness 
is  high  when  compared  to  other  analysis  techniques. 
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3.9.4.13  Link  Analysis 

Description:  This  analytic  tool  is  often  used  as  a  first  t.rp  in 

developing  an  optimized  panel,  workstation,  or  work  area  j 

layout.  It  Is  frequently  used  to  verify  the  adequancy  of  ! 

! 

design  layout.  Its  purpose  Is  to  depict  graphically  the  j 

frequency  and/or  criticality  associated  with  each  of  the  ; 

i 

various  interactions  occurring  between  operator  and  equip¬ 
ment  and/or  between  one  operator  and  another.  The  HE 
analyst  first  starts  with  the  operator  and  equipment 
Interaction  (links)  that  were  established  during  functional 
analysis.  The  data  generated  by  the  OSD's  and  the  correla¬ 
tion  matrix  are  the  major  source  of  link  analysis  aata.  If 
the  link  analysis  is  being  performed  on  a  particular  panel 
layout,  there  may  be  little  of  the  operator-to-operator 
links  involved.  If  the  link  analysis  is  performed  on  a 
system  such  as  the  £-3A  (AWACS)  tactical  compartment, 
however,  the  operator-to-operator  Interactions  are 
extensive. 

There  are  basically  two  types  of  link  analysis  as 
represented  by  the  two  previously  indicated  situations:  the 
panel  layout  and  the  tactical  compartment  (or  other  type  of 
multiple  operators  work  area).  The  term  link  analysis  is 
equally  applicable  to  both  situations.  The  terms  adjacency 
layout  diagrams  and  flow  diagrams  are  sometimes  used  to 
describe  link  analysis  as  It  pertains  to  multiple  operator 
work  areas.  Figure  3.9-14  shows  an  adjacency  layout 
diagram.  The  term  spatial  OSD  (SOSD)  Is  sometimes  used  to 
describe  link  analysis  of  a  console  or  panel  layout.  As 
Its  name  Indicates,  the  SOSD  is  the  OSD  flow  of  data  and 
functional  symbology  superimposed  on  a  picture  of  the  par¬ 
ticular  console  or  panel  or  interest.  Figure  3.9-16  illus¬ 
trates  this.  The  Items  that  are  missing  from  the  OSD 
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Procedure: 


In  this  form  are  the  time  scales,  the  outside  events,  and 
the  columns  and  headings.  All  of  the  symbols  and  links  are 
exactly  as  they  are  indicated  in  Section  3. 9. 4. 9.  Opera¬ 
tional  Sequence  Diagrams.  Whereas  the  OSD  Indicates 
workstation  relationships,  it  does  not  do  this  nearly  as 
well  as  link  analysis  does.  The  spatial  OSD  may  also  be 
used  for  verifying  work  area  layouts  and  the  adjacency 
layout  diagrams  used  to  verify  console  layouts.  However, 
the  latter  situation  is  unusual. 

The  adjacency  layout  diagram  type  of  link  analysis  is 
dependent  on  the  correlation  matrix.  Beginning  with  the 
correlation  matrix  and  a  console  or  area  layout,  all 
interactions  (links)  required  to  perform  a  particular  func¬ 
tional  task  are  examined  in  terms  of  the  frequency  with 
which  they  occur  and  their  criticality.  If  the  criticality 
Is  assigned  a  numerical  value,  it  may  be  multiplied  by  the 
frequency  in  order  to  obtain  a  weighted  link  value.  The 
panel  or  work  area  is  overlaid  with  the  weighted  links 
permitting  a  picture  of  all  the  interactions  taking  place 
within  the  system  being  analyzed.  The  system  design  is 
then  modified  to  shorten  the  distance  between  the  controls 
or  displays  or  workstations  that  are  connected  by  the 
weighted  links. 

There  are  several  variations  in  the  detailed  step  by  step 
procedure  tor  constructing  a  link  analysis  diagram.  The 
variations  are  dependent  on  the  type  of  link  analysis  being 
used  and  the  type  of  layout  being  analyzed,  i.e.,  console 
or  work  area.  Basically,  the  first  step  in  performing  the 
flow  diagram  or  SOSD  analysis  is  to  choose  symbology  for 
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each  of  the  system  functions  being  manipulated  or 
arranged.  It  is  strongly  recommended  that  the  OSD 
symbology  be  used  (see  Figure  3.9-8).  Symbology  for  the 
system  components  Is  not  as  important  as  the  functions 
because  the  drawing  of  the  panel  or  work  area  shows  what 
the  components  ?re  without  the  need  for  any  symbols. 

In  the  case  of  the  adjacency  layout  diagram  special 
symbols,  such  as  circles  for  operators  and  squares  for 
equipment,  may  be  chosen  for  each  of  the  operator /equipment 
categories.  In  this  type  of  analysis  the  frequency  qf  use 
and  criticality  of  links  between  workstations  are  empha¬ 
sized  rather  than  the  flow  sequence.  The  choice  of  line 
coding  for  each  of  the  various  types  of  links  must  be 
made.  There  is  no  standard  for  use  as  a  guide,  but  the 
factors  that  should  be  considered  are  frequency  of  use, 
criticality,  and  type  of  communication  link  (e.g..  Voice, 
TTY).  Often  the  line  width  of  the  link  indicates  either 
the  frequency  of  use  or  the  weighted  value  of  the  link. 

The  frequency  of  use  times  the  criticality  is  the  weighted 
value  of  the  link.  A  criticality  value  of  1,  3,  or  3  is 
recommended.  The  higher  the  total  number  (criticality 
times  frequency),  the  more  significant  the  link.  Often 
this  number  is  labeled  right  on  the  link.  As  previously 
indicated,  the  value  for  the  frequency  of  use  comes  from 
the  correlation  matrix  (Figure  3.9-13)  or  directly  from  the 
OSD's  (Figure  3.9-9). 

In  either  case,  the  last  step  in  the  technique  is  to  draw 
on  an  overlay,  or  to  draw  directly  onto  the  design  layout, 
the  links  and  symbols  selected.  It  is  important  to  have 
selected  a  drawing  that  Is  to  scale.  If  the  SOSO  technique 
is  being  used,  the  analyst  starts  at  the  beginning  of  the 
SOSD  with  the  OSD  symbology  and  proceeds  to  the  completion 
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of  the  total  major  task  (see  Figure  3.9-15).  If  the 
adjacency  layout  diagram  technique  is  being  used,  the  HFE 
analyst  starts  with  the  operator  who  appears  to  be  the 
busiest.  He  places  the  related  components  arour.d  the 
operator,  moving  them,  as  necessary,  to  minimize  link 
crossings  (If  significant)  and  to  shorten  linx  lengths,  es¬ 
pecially  those  with  high  weighted  link  values.  It  should 
be  emphasized  that  additional  changes  undoubtedly  will  be 
required  once  the  system  is  constructed  in  the  form  of  full 
scale  mockups  or  as  prototype  hardware.  Regardless  of  a 
paper  analysis,  the  system  requires  an  interactive  review. 

Use/Validity:  Table  3.9-1  lists  the  applications  or  outputs  for  which 

link  analysis  data  may  be  used.  Table  3.9-4  indicates  the 
comparison  between  link  analysis  and  the  numerous  other 
techniques.  In  summary,  the  table  indicates  that  link 
analysis  should  be  used  during  the  first  or  middle  phases 
of  a  program.  It  is  of  average  complexity  to  perform  as 
compared  to  other  analysis  techniques.  It  should  be  used 
for  detailed  analysis  and  like  the  correlation  matrix  much 
of  its  purpose  is  to  analyze  several  nearly  simultaneous 
tasks.  The  time  taken  to  develop  a  link  analysis  is 
average.  It  may  be  used  for  presentation  of  data  to 
managers  but  is  best  used  by  analysts.  Its  cost  is  average 
and  cost  effectiveness  if  slightly  better  than  average  when 
compared  to  other  analysis  techniques. 

3.9.4.14  Computer-Aided  Function  Allocation  and  Evaluation  System  (CAFES) 

The  magnitude  of  human  engineering  tasks  is  frequently  too 
great  for  manual  completion  in  compliance  with  design/de¬ 
velopment  scheduling  requirements,  forcing  either  minimal 
consideration  or  heavy  reliance  on  professional  experience 
and  judgment.  There  is  need  for  an  integrated,  interactive 
system  for  more  effective  human  factors  engineering 
efforts,  to  expedite  time  consuming  HFE  task  elements  in 
data  retrieval  and  processing.  In  this  regard, 
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properly  designed  computer  programs  can  extend  the  capabil¬ 
ities  of  the  human  factors  engineer.  This  section  de¬ 
scribes  such  a  system  for  improving  and  expediting  the  HFE 
analysis  process.  It  summarizes  the  concepts  of 
computer-aided  techniques  for  human  engineering  support  to 
Navy  systems  development  order  a  program  called.  CAFES 
(Computer-Aided  Function  A1 location  and  Evaluation 
System).  CAFES  is  a  design  support  system  based  on  human 
engineering  methods,  computer  aids,  human  performance  data, 
and  a  data  management  system. 

CAFES  offers  a  number  of  computer  aids  to  HFE  that  can  oe 
applied  throughout  system  development.  When  fully 
completed,  validated,  and  implemented,  it  will  provide  for 
a  systematic  integration  of  computer  and  engineering 
capabilities.  As  system  development  progresses,  CA^ES  can 
be  used  in  initial  development  and  exercised  repeatedly 
throughout  development  to  assist  in  updating  requirements 
analysis;  system  trade-offs;  definition  of  design  criteria; 
crew  systems  design;  procedures  development;  test  and  eval¬ 
uation  planning;  training  and  maintenance  system 
development;  and  operational  evaluation. 

The  CAFES  submodels  include: 

a)  Data  Management  System  (DMS) 

b)  Function  Allocation  Model  (FAM) 

c)  Workload  Assessment  Model  (WAM) 

d)  Computer-Aided  Crew  Station  Design  Model  (CAD) 

e)  Crew  Station  Geometry  Evaluation  Moael  (CGE) 

The  separate  CAFES  models  are  interrelated  and  can  be 
interdependent,  as  the  inputs  to  some  models  can  be  the 
outputs  from  others.  For  example,  a  workload  analysis 
(WAM)  can  evaluate  candidate  function  allocations  (FAM;  and 
integrate  necessary  task  sequence/timeline  data  as  a  pre¬ 
requisite  to  preliminary  design  development  (CAD).  This 
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Integration  of  the  various  models  Into  one  coherent  system 
provides  an  efficient  exchange  of  data  between  submodel  el¬ 
ements  as  well  as  use  of  common  data.  Iterative  analyses 
responsive  to  system  or  concept  changes  are  also 
facilitated  by  the  integration. 

CAFES  can  be  applied  at  a  gross  level  during  early  system 
concept  formulation  when  system  detail  is  usually  sketchy, 
or,  with  numerous  assumptions,  at  a  detailed  level.  As 
system  development  progresses,  the  ratio  of  system  detail 
to  system  assumptions  improves  considerably  and  CAFES 
analyses  can  be  carried  out  to  much  greater  detail.  This 
will  permit  updating  of  analyses  to  reflect  changes  from 
later  submodels  and  current  HFE  status  throughout  the  de¬ 
velopment  cycle. 

The  following  paragraphs  summarize  the  concept  for  each 
CAFES  submodel.  The  CAFES  executive  and  management  system, 
DMS,  and  the  FAM  and  WAM  submodels  are  discussed  in  this 
analysis  paragraph.  CAD  and  CGE  are  presented  in  the  para¬ 
graph  on  design  techniques.  More  complete  descriptions  are 
contained  in  References  34  through  40.  The  application  of 
each  separate  CAFES  model  in  HFE  is  discussed  under  each 
model  subsection,  however,  one  use  of  CAFES  is  in  the  inte¬ 
gration  of  the  models  to  produce  data  and  analysis  required 
during  new  systems  development.  CAFES  model  relationships 
are  illustrated  in  Figure  3.9-16.  The  interactive  applica¬ 
tions  of  these  models  can  produce  all  the  various  CAFES 
results. 

For  example,  the  workload  analysis  by  WAM  may  suggest  a  re¬ 
exercise  of  the  functions  allocations  in  FAM  to  evaluate 
different  allocation  versions;  or  CGE  results  may  suggest  a 
change  in  basic  configuration  layout,  to  be  run  on  the 
CAD.  Consequently,  the  fully  integrated  capability  of  the 
CAFES  method  will  be  realized  when  all  submodels  are  com¬ 
pleted  and  interrelated. 
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Figure  3.9-!6:CAFES  Model  Relationships 


CAFES  Oata  Management  System  (DM$) 


One  of  the  major  elements  support!. ig  the  CAFES  system  and 
all  CAFES  subsystems  is  the  Data  Management  System  (QMS). 
While  perhaps  peripheral  to  the  main  flow  of  the  CAFES 
operation,  it  provides  baseline  data  for  all  models.  DMS 
serves  three  purposes.  First,  it  provides  a  unified  system 
for  storing,  updating  and  retrieving  all  data  needed  by 
CAFES.  Second,  as  the  CAFES  executive,  it  has  an  operating 
interface  with  all  subsystems  and  is  used  In  all  models. 
Finally,  it  is  under  direct  control  of  the  analyst  for  use 
in  either  input  or  output  of  CAFES  data. 

The  objectives  of  the  data  management  system  are  a)  to  pro¬ 
vide  rapid  access  to  standardized  data  relative  to  opera¬ 
tional  and/or  proven  system  concepts  for  use  by  both  the 
CAFES  submodels  and  the  HFE  analyst,  b)  to  allow  for 
amalgamation  of  data  commensurate  with  a  given  level  of 
system  definition  in  a  rapid  and  easy  manner,  and  c)  to 
provide  an  information  storage  scheme  sufficiently  general 
to  handle  the  diverse  data  requirements  of  the  submodels. 
Major  functions  performed  by  the  OMS  are: 

a)  Oata  Input  and  Storage:  Provides  means  to  enter 
and  file  information  into  the  computer,  including 
input  format,  data  addressing,  storage  allocation, 
etc. 

b)  File  Modification:  Provides  means  to  add,  delete, 
or  substitute  data  in  storage. 

c)  CAFES  executive:  Provides  executive  function  to 
execute  CAFES  submodels,  transfer  data  to  and  from 
files,  generate  reports,  etc. 

d)  Error  Diagnostics:  Provides  means  for  determining 
and  reporting  the  cause  of  output  errors  and  .  un 
interruptions. 
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e) 


Report  Generation:  Provides  means  for  retrieving 
information  from  the  computer.  Including  report 
type  (e.g.,  tabular  or  graphical),  report  format, 
labeling,  etc. 

CAFES  Function  Allocation  Model  (FAM) 

The  FAM  is  a  collection  of  computerized  algorithms  that 
will,  in  conjunction  with  the  QMS  and  HFE  analyst,  derive 
and  process  various  alternatives  for  allocating  functions 
to  operators  or  equipment.  The  genera1  objectives  of  the 
FAM  are  to  identify  and  organize  system  functions  to  an 
allocatable  level,  and  to  identify  and  to  rank  order  func¬ 
tion  allocation  schemes  (by  performance  effectiveness)  that 
satisfy  mission  requirements. 

The  FAM  works  from  a  user-specified  list  of  system 
functions,  performance  data  and  allocation  candidates  in  an 
iterative  process;  a)  to  predict  overall  system  effective¬ 
ness  (probability  of  mission  success)  and  b)  to  generate 
crew  operational  procedures  for  detailed  evaluation  of 
promising  allocation  candidates.  Use  of  the  FAM  for  evalu¬ 
ating  allocation  candidates  is  straightforward.  For  the 
initial  application  on  a  proposed  aircraft  system,  the  HFE 
analyst  extracts,  'modifies,  and  assembles  system 
functions.  To  the  extent  that  functions  are  similar  to 
those  contained  in  the  DMS,  a  primary  data  file  can  be 
rapidly  assembled  and  structured.  If  the  FAM  or  other 
CAFES  submodel  has  been  used  previously  on  the  particular 
aircraft  system,  data  may  be  available  also  from  these, 
e.g.:  the  Workload  Assessment  Model  (WAM)  for  function  al¬ 
location  processing.  The  FAM  output  is  checked  by  the  HFE 
analyst  for  consistency  with  system  requirements.  If  allo¬ 
cations  are  consistent,  the  user  modifies  the  FAM  input 
data  and  reruns  FAM.  The  major  FAM  functions  are: 


a)  Mission  Evaluator:  Computes  probability  of  overall 
mission  success  for  various  function  allocation 
candidates.  Success  probabilities  for  specific 
mission  objectives  can  also  be  computed. 

b)  Procedure  Generator:  Derives  data  for  use  with  op¬ 
erational  sequence  diagrams  and  procedure 
statistics  based  on  function  allocations,  task 
priorities,  procedure  constraints,  etc. 

Given  preliminary  functions  allocation  candidates,  the  task/ 
workload  process  described  later  is  applied  to  appraise 
needs  for  reallocation  and  refinement.  System  effectiveness 
is  predicted  on  the  basis  of  operator  and  machine  perfor¬ 
mance  in  terms  of  task  error  rates  and  task  execution 
times.  Operational  procedures  are  derived  according  to  user 
specified  rules  and  constraints  on  the  mission  tasks.  From 
FAN  outputs,  operational  sequence  diagrams  can  be  construc¬ 
ted  for  selected  allocation  schemes.  Table  o.'T-l  indicates 
the  applications  or  outputs  of  FAM. 

CAFES  Workload  Assessment  Model  { WAM ) 

The  WAM  considers  the  human  performance  aspect  of 
man-machine  function  allocation  schemes  on  a  time  and 
cumulative  cask  basis  to  determine  whether  mar.  can  perforin 
all  of  the  tasks  derived  from  the  allocated  functions.  The 
submodel  uses  a  timeline  of  mission  tasks  and  determines 
those  periods  when  man  is  overloaded  in  terms  of  time 
available  versus  time  required  to  do  all  tasks,  indicating 
the  necessity  for  a)  task  rescheduling,  b)  reallocation  of 
the  function  (or  portions  of  it)  to  equipment  or  additional 
crew,  or  c)  modification  of  the  system  requirements.  Work¬ 
load  can  be  analyzed  for  each  operator  in  a  crew  to 
determine  how  changes  in  task  allocations  will  aUeviate 
overloading  conditions. 
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WAM  Is  based  on  workload  variations  In  each  performance 
channel  (e.g.,  eyes,  hands,  feet).  WAM  generates  bargraph 
and  histogram  plots  of  workload  data  for  use  by  the  HFE 
analyst  so  that  results  may  then  be  visually  scanned  to 
find  heavy  workload  situations.  If  possible,  task  schedul¬ 
ing  can  then  be  moved  to  other  time  periods  to  reduce  ex¬ 
cessive  workload.  WAM  also  provides  an  option  for  automat¬ 
ically  shifting  tasks  to  equalize  workload.  Figure  3.9-17 
illustrates  samples  of  WAM  histogram  outputs.  Table  3.9-1 
indicates  the  applications  or  outputs  of  WAM. 


SAINT 

SAINT  (Systems  Analysis  of  Integrated  Networks  of  Tasks)  Is 
a  computer-aided  technique  that  is  useful  for  analysis  of 
task/activity  networks  (Ref.  41,  Wortman,  1977).  SAINT  has 
been  developed  by  the  Air  Force  Aerospace  Medical  Research 
Lab  along  with  Purdue  University  and  Pritsker  and 
Associates.  It  is  a  modeling  and  simulation  technique  de¬ 
veloped  to  assist  in  the  design  and  analysis  of  complex 
man-machine  systems.  SAINT  consists  of  a  symbol  set  for 
modeling  systems  and  a  computer  program  for  analyzing  such 
models.  SAINT  provides  the  conceptual  framework  for  repre¬ 
senting  systems  that  consist  of  discrete  task  elements, 
continuous  state  variables,  and  interactions  between  them. 
While  SAINT  was  designed  for  modeling  manned  systems  in 
which  human  performance  is  a  major  concern,  it  Is 
potentially  applicable  to  a  broad  class  of  systems-  those 
in  which  discrete  and  continuous  elements  are  to  be 
portrayed  and  quantified  and  whose  behavior  exhibits 
time-varying  properties.  SAINT  provides  a  mechanism  for 
describing  these  dynamics  so  a  systematic  assessment  can  be 
made  of  the  relative  contribution  system  components  made  to 
overall  system  performance. 
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Figure  3.0-17:  Sample  WAM  Hand  Workload  Histograms 
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Systems  are  created  as  graphical  networks  of  task  activi¬ 
ties  with  which  one  or  more,  operators  Interact.  Each  task 
In  a  network  Is  described  as  to  now  Its  performance  affects 
the  overall  system  and  how  \i  is  related  to  other  tasks 
within  the  system.  The  graphical  operator /task  analysis 
system  description  is  entered  Into  the  SAINT  computer  pro¬ 
gram  for  automated  performance  assessment.  Employing  Monte 
Carlo  techniques,  SAINT  permits  the  simulation  of  probabi¬ 
listic  and  conditional  task  performance  descriptions  and 
precedence  relationships.  It  also  permits  the  collection 
of  statistical  estimates  of  system  performance.  Another 
major  capability  of  the  program  is  the  system  characteris¬ 
tics  in  response  to  system-internal  or  external  simulated 
events. 

By  design,  the  SAINT  technique  does  not  require  the  user  to 
perform  any  computer  programming  although  experience  in 
this  field  is  extremely  helpful.  Users  are  assumed  to  be 
knowledgeable  of  task  analysis.  The  results  of  a  task 
analysis  are  used  as  the  inputs  to  the  SAINT  computer 
program.  The  output  of  SAINT  consists  of  task  and  mission 
performance  estimates. 


.16  TUA-1 

The  acronym  TLA- 1  derives  from  "Time  Line  Analysis  program 
-  model  one".  It  is  generally  referred  to  as  TLA-1  rather 
than  the  complete  descriptive  title.  As  its  complete  title 
indicates,  TLA  is  a  time  line  analysis  model.  It  is  also 
used  for  workload  analysis  in  a  manner  similar  to  the  work¬ 
load  techniques  presented  in  this  section.  It  is  strongly 
oriented  towards  cockpit  analysis  although  it  is  easily 
adaptable  to  any  crew  station. 
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!  The  TLA-1  computer-aided  analysis  technique  Is  Initiated  by 

|  the  preparation  of  scenarios  and  crew  task  data.  The  HFE 

t  analyst  generates  scenario  data  from  sources  such  as  flight 

|  plans,  aircraft  performance  data,  and  aircraft  operations 

|  manuals.  If  the  analysis  is  for  a  completely  new  aircraft, 

|  the  data  must  come  from  existing  similar  aircraft.  Since 

!  operator  tasks  are  the  basic  work  units  from  which  all 

TLA-1  crew  workload  statistics  are  derived,  they  must  be 
identified  for  every  control,  display,  and  communication 
link.  It  is  possible  to  catalog  over  2,000  tasks  for  one 
analysis  effort.  The  tasks  are  categorized  by  aircraft 
;  subsystems.  Each  task  description  contains  a  task  code 

number,  a  task  description/name,  task  duration  time  and  the 
channel  activity  (left  hand,  rignt  hand,  external  vision, 
internal  vision,  cognition,  etc.). 

i 

After  the  scenarios  and  tasks  have  been  defined,  the 
analyst  develops  the  detailed  task  sequence  required  to 
execute  the  scenario.  Worksheets  are  used  for  this 
detailing.  In  the  process  of  filling  in  the  details  on  the 
worksheet,  the  HFE  analyst  specifies  all  the  data  that  will 
be  entered  onto  the  various  input  data  coding  forms. 

The  next  step  is  the  input  data  coding.  Each  of  the  six 
sets  of  input  data  has  a  fixed-format  coding  form  that  the 
analyst  uses.  These  data  coding  forms  are  for  subsystems 
data,  task  data,  events/procedures,  phase  data,  mission 
data,  and  output  report  and  plot  request  coding. 

One  of  the  most  powerful  features  of  TLA-1  is  the  wide 
variety  of  workload  analysis  data  formats  that  are 
available.  There  are  six  digital  reports  and  four  data 
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plots  that  can  be  requested.  By  specifying  various  vari¬ 
ables  for  each  of  these  output  formats,  there  are  literally 
thousands  of  data  records  that  can  be  selected  for  output 
for  a  mission.  Obviously,  not  every  conceivable  report  and 
plot  will  be  requested  at  any  one  time. 

Standard  sets  of  reports  and  plots  have  been  defined  that 
can  be  specified  by  number.  The  Items  in  these  standard 
report  sets  have  been  selected  to  provide  a  general  visi¬ 
bility  of  the  workload  situation  for  a  scenario.  As  high 
workload  problems  are  isolated,  the  analyst  can  be  more 
selective  of  the  output  types  and  exercise  tighter  control 
over  the  variables  so  that  successive  data  outputs  can 
expose  the  nature  of  the  workload  problems  in  more  detail. 

The  TLA-1  computer  program  is  divided  into  the  executive, 
input,  processor,  and  output  modules.  The  executive  module 
processes  all  control  cards  and  initiates  the  other  three 
modules.  All  mission  data  are  input  through  the  input 
module  and  output  to  an  external  permanent  file.  The  pro¬ 
cessor  performs  all  the  calculation  functions  and  outputs 
the  results  to  an  external  file.  The  input  to  the  proces¬ 
sor  comes  from  the  data  stored  by  the  input  module.  The 
output  module  inputs  report  requests  and  acts  to  produce 
the  requested  reports  using  the  data  from  the  two  external 
files  created  by  the  input  module  and  the  processor 
module.  There  may  be  up  to  three  sets  of  external  files 
(different  configurations  of  the  same  mission)  input  to 
create  some  reports.  Outputs  from  the 
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TLA-1  program  are  to  tape,  printer,  and  plotter.  A  tape  is 
used  to  store  the  mission  data  input  and  the  processed  data 
for  later  use  by  the  report  generation  function.  The  tape 
consists  of  two  files.  The  first  contains  the  mission  data 
Input.  The  second  contains  the  processor  output  used  by 
the  report  generator  function. 

The  output  to  the  printer  consists  of  seven  reports: 

a)  Mission  Scenario 

b)  Crewman  Workload  Profile 

c)  Crewman  Workload  Summary  Statistics 

d)  Task  Channel  Activity 

e)  Subsystem  Activity 

f)  Subsystem  Activity  Sunmary 

g)  Task  List 

The  plotter  output  consists  of  a  workload  summary,  a 
channel  activity  summary,  a  workload  histogram,  and  a  mis¬ 
sion  timeline.  Figure  3.9-18  is  a  sample  channel  activity 
summary  and  Figure  3.9-19  a  sample  workload  histogram  plot. 

Table  3.9-1  Indicates  the  applications  or  outputs  of  TLA-1 
compared  to  the  outputs  of  other  analysis  techniques.  Ad¬ 
ditional  information  on  TLA-1  is  available  In  Reference  42 
(Miller,  1976). 
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Figure  3.9-19:  Sample  TLA- I  Workload  Histogram  Plot 


3.9.5  Design 


The  purpose  of  this  activity  is  to  provide  a  system  man-machine  design 
which  incorporates  all  necessary  HE  design  criteria.  The  man-machine 
interface  design  is  not  limited  to  portions  of  system  equipment,  but  in¬ 
cludes  software  design,  procedures,  work  environments,  and  facilities  as¬ 
sociated  with  the  system  functions  requiring  personnel  interaction.  This 
activity  is  accomplished  by  converting  the  results  of  the  analysis 
activity  into  HE  and  Biomedical  design  criteria.  It  is  heavily  dependent 
on  the  selection  of  applicable  MIl-STD-1472  design  criteria. 

In  order  to  develop  and/or  apply  appropriate  HE  design  criteria  to  the 
system  design,  a  concerted  HE  design  effort  must  be  accomplished.  Many  of 
the  most  useful  design  aids,  tools,  or  techniques  which  are  appropriate 
for  use  of  HE  are  presented  in  the  following  sections.  Depending  on  the 
nature  of  the  program,  only  a  portion  of  them  would  normally  be  used. 
Sufficient  time  or  HE  effort  does  not  exist  to  use  all  of  the  techniques 
for  a  single  program.  Much  of  the  data  presented  are  also  organized  into 
tabular  form  in  Table  3.9-7.  By  listing  the  techniques  in  one  chart  they 
may  be  easily  compared  for  possible  selection  and  use.  Reference  43 
(Roebuck,  1975)  provides  additional  information  of  several  of  the  design 
techniques  and  tools  including  vision  plots,  reach  envelopes,  mockups,  and 
manikins. 


3.9.5. 1  Design  Criteria  Checklist 

Description:  The  checklist  Is  a  series  of  equipment  and  facilities  design 
requirements  of  criteria  taken  from  human  engineering 
standards,  e.g.,  MIl-STD-1472,  handbooks  and  guides.  Often, 
during  the  early  stages  of  a  program,  a  checklist  is  devel¬ 
oped  by  HF  engineers  for  that  particular  program.  Design 
criteria  which  would  be  applicable  to  the  particular  program 
are  extracted  from  the  various  standards  and  handbooks  and 
listed  in  a  program  unique  checklist.  The  checklist  may  be 
divided  up  into  sections  or  categories  of  design  criteria 
corresponding  to  major  equipment  or  facilities 
characteristics.  These  categories  might  be  visual  displays, 
audio  displays,  controls,  etc.  The  checklists  generally 
have  a  space  to  the  right  of  each  listed  item  of  design 
criteria.  This  space  is  divided  into  three  columns: 
compliance,  noncompliance,  and  not  applicable.  Figure 
3.9-20  is  a  sample  page  from  the  checklist. 

Frocedure:  The  HFE  evaluator  reads  the  item  of  criteria,  observes  the 

item  of  hardware  (or  mockup  or  drawing),  and  checks  the  ap¬ 
propriate  space  for  applicability  and  compliance.  Many 
checklists  provide  additional  space  to  include  comments  as 
to  the  reason  for  noncompliance  or  other  remarks  appropriate 
to  the  listed  design  criteria  item. 

The  HFE  evaluator  should  initiate  the  use  of  the  C’.ecklist 
with  at  least  some  knowledge  of  the  purpose  or  function  of 
the  design  item  being  evaluated.  He  must  have  a  good 
working  knowledge  of  the  checklist  criteria  which  he  will  be 
using.  He  should  determine  if  the  item  of  hardware  has  had 
any  previous  checklists  completed  on  it,  even  if  the 
hardware  was  only  in  drawing  form  at  the  time.  The  more 
formal  test  and  evaluation  procedure  will  occur  when  the 
item  being 
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Figure  3.9-20:  Sample  MIL-STD-1472  Checklist  Page 


evaluated  is  at  least  1-,  *.'.e  prototype  hardware  stage  of 
development.  Less  formal  checklist  test  and  evaluation  may 
take  place  with  hardware  drawings  or  possibly  mockups.  In 
any  case,  the  evaluation  should  take  place  on  a 
noninterference  basis,  i.e.,  the  gathering  of  the  checklist 
data  should  not  interfere  with  the  conduct  of  any  other  test 
aspects.  The  use  of  the  checklist  is  essentially  a  static 
operation,  as  opposed  to  a  dynamic  test  which  requires  ob¬ 
servation  of  operators  performing  their  tasks  and  equipment 
properly  responding  to  their  manipulation. 

The  checklist  evaluation  will  result  in  a  verification  of 
the  fact  that  the  design  item  meets  all  pertinent  HE  design 
criteria.  If  some  design  criterion  is  found  not  in  proper 
compliance,  then  this  information  will  be  provided  to  design 
engineering  personnel.  In  some  situations,  there  may  be 
satisfactory  rationale  as  to  why  an  item  of  hardware  does 
not  or  should  not  meet  the  HE  design  requirements.  In  this 
case,  a  request  for  deviation  to  HE  design  criteria  may  be 
submitted  to  the  Air  Force  system  program  office  for  their 
approval . 

Use/Validity:  This  technique  is  used  more  often  than  any  other  to  evaluate 
design  hardware.  It  is  an  excellent  way  to  gather  quickly 
qualitative  aata  on  system  hardware  components.  However,  in 
order  to  he  of  real  value,  there  must  be  considerable  detail 
contained  within  the  checklist.  Depending  upon  how  the 
checklist  is  structured,  the  amount  of  detail  required  for 
review  can  extend  the  time  required  to  perform  the 
checklist.  Use  of  the  check  list  requires  more  knowledge  of 
basic  HE  design  criteria  than  system  performance 
requirements. 


The  disadvantages  associated  with  the  use  of  toe  checklists 
are  that  they  produce  binary  data;  the  design  criteria  being 
verified  is  either  in  compliance  or  not.  However,  many  cri¬ 
teria  items  have  the  potential  for  an  exact  quantitative 
evaluation;  thus  considerable  data  will  be  unrecorded.  The 
checklist  is  used  for  evaluation  of  hardware  only.  In  its 
present,  generally  agreed-to  formats,  the  checklist  will  not 
evaluate  personnel  skills,  quantities,  training,  technical 
publications,  etc. 

The  use  of  this  particular  technique  is  strongly  advised  for 
both  design  and  T&E  program  activities.  If  not  used,  there 
is  significant  risk  that  lack  of  critical  design  compliance 
requirements  will  be  overlooked. 

3. 9. 5. 2  Drawings 

Description:  Engineering  sketches  und  drawings  are  precise  outline 

drawings  (usually  void  of  shading)  used  to  provide  informa¬ 
tion  as  to  the  design  of  the  item,  facility,  or  subassembly 
which  is  a  component  or  part  of  the  total  system.  By  a 
logical  procedure  of  depicting  related  drawing  "views", 
intricate  and  complicated  shapes  are  clearly  shown.  Exact 
and  detailed  sizes  are  provided  without  ambiguity. 

Individual  parts  are  identified  for  assembly  and  are  located 
in  the  assembly  in  their  correct  functional  position.  In 
addition,  descriptive  notes  provide  information  as  to 
materials,  finishes,  and  directions  for  manufacture  and 
assembly. 

Often  engineering  drawings  are  referred  to  as  sketches. 

This  is  only  because  of  their  Intended  lack  of  contractor  or 
customer  sign-off  approval.  They  are  in  every  other  respect 
similar  to  engineering  drawings.  Engineering  drawings  or 
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Procedure: 


sketches  of  interest  to  HE  personnel  may  be  further 
categorized  as  hardware  drawings,  workspace  layout  drawings, 
console  drawings,  and  panel  drawings.  Console  drawings,  in 
particular,  should  contain  Information  as  to  the  man-machine 
interface,  for  example,  the  seat  reference  point  (SRP)  and 
eye  reference  point  ( ERP)  should  be  indicated.  Interface 
control  drawings  (ICD's)  are  another  type  of  drawing  that 
should  require  HE  review.  As  their  name  implies,  these 
drawings  are  used  to  describe  and  to  eventually  control 
proposed  interfaces  between  components,  subsystems,  or 
different  contractor's  equipment  items.  Vision  plots  (Ref. 
Figure  3.9-21)  and  reach  envelopes  (Ref.  Figure  3.9-22)  are 
two  additional  types  of  drawings  of  particular  interest  to 
HE. 


Generally,  engineering  drawings  are  used  by  HE  personnel  to 
review  the  design  concepts.  However,  the  HE  group  may 
actually  prepare  engineering  drawings  for  their  own  use  and 
the  use  of  others.  The  development  of  engineering  drawings 
by  HE  are  predicated  on  the  data  necessary  to  initiate  the 
drawings  including  the  drawing  equipment  and  the  skills  of 
engineers,  draftsmen,  or  industrial  designers. 

The  preparation  of  workspace  layout  drawings  requires  sxill 
in  descriptive  geometry.  The  HE  analyst  must  be  able  to 
project  views  and  cross  sections  of  the  worxspace  geometry 
and  the  human  subject  into  various  auxiliary  planes  which 
often  are  not  parallel  to  the  normal  planes  of  the  three- 
view  or  the  graphic  engineering  drawings.  Also,  for 
purposes  of  visual  clarity  and  understanding,  perspective 
drawing  techniques  should  be  understood  and  used.  The 
ability  to  mentally  visualize  the  geometry  of  workspace 
layouts  and  to  accurately  prepare  drawings  depicting  the 
Interface  relationships  can  save  time  and  effort  during 
mockup  studies. 
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More  normally,  HE  personnel  use  engineering  drawings  devel¬ 
oped  by  project  design  personnel.  They  must,  of  course,  be 
sufficiently  knowledgeable  of  standard  (Air  Force  and 
contractor)  drawing  practices  to  understand  the  information 
being  presented.  HE  design  criteria  checklists  (Ref.  Figure 
3.9-20)  may  be  used  along  with  fractional  scale  plastic 
manikins  to  insure  the  HE  adequacy  of  the  design.  Once  this 
adequacy  is  assured,  the  drawings  should  be  signed-off  to 
indicate  HE  design  application  approval. 

Use/Val idlty-  .  specialists  have  prepared  the  engineering  drawings,  it 
assured  that  the  drawings  incorporate  all  appropriate  HE 
design  criteria  and  that  HE  sign-off  (as  discussed  in  Sec¬ 
tion  3.4.1)  is  automatically  provided.  If  the  drawings  are 
prepared  by  other  project  engineering  personnel,  HE  should 
thoroughly  review  them  to  insure  the  inclusion  of  appropri¬ 
ate  HE  design  criteria.  The  MIl-STD-1472  checklist  should 
be  used  at  this  time.  Completion  of  the  checklist  will  pro¬ 
vide  justification  for  HE  sign-off  (or  lack  of  same)  for  the 
drawings. 

In  addition  to  HE  design  verification,  engineering  sketches 
and  drawings  specify  the  detailed  design  of  the  hardware 
item.  They  provide  a  baseline  configuration  record  (Ref. 
Sections  3. 3. 1.3  and  3.9.8),  they  provide  inputs  to  initiate 
mockup  construction,  and  they  provide  manufacturing  with  the 
necessary  data  from  which  to  produce  the  hardware  product. 
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3.'}. 5. 3  Visibility  Diagram 

Description:  The  vision  plot  or  visibility  diagram  Is  a  special  drawing 
to  show  the  vision  envelope  of  specific  system  operators. 

An  analysis  of  their  vision  envelope  capabilities  can  be 
provided  by  multiple  views  of  the  operator  In  front  of  the 
console  or  other  Instruments  and  controls.  However,  rather 
than  showing  the  side,  top,  and/or  front  views,  the  visibil¬ 
ity  diagram  shows  the  actual  view  from  the  operator's  eye 
(eye  reference  point,  ERP).  Figure  3.9-21  is  a  sample 
cockpit  visibility  diagram.  As  can  be  seen  from  this 
d'agram,  the  envelope  Is  a  plot  of  angles  both  to  the  left 
and  right  of  the  operator's  sagittal  plane  (directly 
forward)  and  up  and  down  from  the  horizontal  plane  through 
the  ERP. 

Procedure:  Visibility  diagrams  are  developed  in  accordance  with  specif¬ 

ic  procedures  such  as  those  detailed  in  MIL-STD-850  (Ref. 
44).  The  HE  analyst  or  draftsman  preparing  tne  drawings 
works  from  the  two  or  three  view  orthographic  drawings  of 
the  operator  work  station  (e.g.,  flight  deck  or  cockpit). 
Through  descriptive  geometry  techniques,  he  measures  the 
angles  from  the  ERP  to  significant  Items  shown  In  the 
orthographic  drawings.  Windows,  Instruments  or  controls  are 
generally  the  primary  Items  of  interest  in  the  visibility 
diagrams.  The  angles  to  several  points  on  each  of  the  sig¬ 
nificant  Items  are  measured  and  plotted  In  order  to  approxi¬ 
mate  the  shape  of  the  item.  All  straight  lines  shown  on  the 
orthographic  projection  (with  the  exception  of  vertical 
lines  and  lines  within  the  horizontal  plane  through  ERP) 
will  be  plotted  as  curved  lines.  Straight  lines  below  the 
horizontal  plane  will  curve  up,  and  above  the  plane  will 
curve  down. 
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Use/ Val 1 d 1 ty;  Visibility  envelopes  are  useful  to  determine  what  operators 
can  and  cannot  see.  Their  use  in  cockpit  or  flight  deck  de¬ 
sign  is  extremely  critical  to  determine  where  window  posts 
are  located  in  reference  to  the  pilot's  runway  vision  at 
various  landing  approach  geometries.  Whereas  new  aircraft 
design  aerodynamic  considerations  tend  to  dictate  flat  angle 
smooth  surfaces  around  the  aircraft  cockpit  area,  these 
considerations  cannot  violate  the  pilot's  minimum  vision  re¬ 
quirements  as  described  in  military  and  FAA  specifications. 
The  visibility  diagram  provides  a  technique  for  making  the 
specification  comparison.  It  further  provides  a  record  of 
the  system  design  and  generally  avoids  the  cost  of 
preliminary  mockups  which  would  otherwise  be  constructed 
just  to  evaluate  operator  vision. 

3. 5. 4  Reach  Envelopes 

Reach  envelope  drawings  describe  the  envelope  within  which 
controls  must  be  placed  in  order  to  be  successfully  reached 
by  the  subject  operator.  Until  recently,  the  operator  has 
generally  been  described  as  one  with  a  5th  percentile  func¬ 
tional  reach.  Recent  bimodel  male-female  populations  may 
not  include  sufficient  data  to  calculate  the  lower  limit 
percentile  for  determining  the  desired  reach  envelope.  The 
envelopes  vary  greatly  for  the  5th  percentile  operator  for 
known  male  populations.  This  is  because  of  variations  of 
seat  design  and  shoulder  and  lap  constraints  if  the  operator 
subject  is  seated.  Reach  envelopes  are  also  developed  and 
used  for  overhead  reach. 

The  procedure  for  developing  reach  envelopes  is  simply  to 
modify  or  adapt  existing  data  or  to  develop  new  data.  Func¬ 
tional  reach  is  always  the  parameter  of  main  interest. 
Measurements  are  made  with  the  subject's  thumb  and 
forefinger  tips  pressed  together.  Secondary  parameters  such 
as  shoulder  height  are  also  of  interest  and  combine  with 
functional  reach  to  provide  the  total  reach  envelope  data. 
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Information  showing  appropriate  combined  reach  data  are 
available  In  OH  1-3  and  a  few  other  sources.  If,  because  of 
peculiarities  in  the  particular  new  system  seat  and  the  op¬ 
erator  restraint  system.  It  Is  not  possible  to  use 
previously  developed  data,  then  new  data  can  be  developed. 
This  will  require  the  gathering  of  appropriate  size  and 
number  of  subjects  to  match  the  population  and  the  seat  to 
be  used  In  the  new  system.  Reach  capability  data  must  be 
taken  for  each  of  the  subjects  under  various  conditions, 
such  as  a  pressure  suit,  seat  back  angle,  and  shoulder 
restraint,  and  in  various  directions  and  heights  in  relation 
to  the  seat  reference  point  (SRP)  or  ground  reference 
plane.  Once  the  data  are  obtained,  statistical 
distributions  of  reach  data  may  be  plotted  and  a  percentile 
curve  or  statistical  estimate  may  be  selected  and  prepared. 
The  envelope  drawings  are  then  plotted  and  overlaid  onto  the 
console  or  cockpit  drawings.  The  SRP  or  other  hardware 
datum  reference  is  necessary  to  establish  where  the  reach 
envelope  should  be  located.  Examination  of  two  or  more 
different  orthographic  views  of  the  control  panel  hardware, 
which  a^e  overlaid  by  the  envelopes,  will  determine  if  the 
necessary  controls  are  within  the  operator's  reach  or  if  the 
controls  and  operator  must  be  moved  closer  together. 

Reach  envelope  drawings  are  Important  to  proper  console 
design,  particularly  if  the  console  is  large  with  side 
wraparound  panel  areas  or  vertical  panel  areas  which  project 
above  the  eye  reference  point  (ERP)*  Proper  use  of  reach 
envelope  drawings  will  save  later  motkup  construction 
effort.  Engineering  drawings  and  sketches  may  be  validated 
prior  to  the  use  of  mockups  and  prototype  hardware.  If 
properly  presented,  reach  envelopes  may  be  easily  understood 
by  non-HE  personnel  and  can  be  very  useful  as  a  part  of 
hardware  design  review  presentations.  Figure  3.9-ZZ  illus¬ 
trates  a  sample  reach  envelope  drawing. 
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CONSOLE  OUTLINE 


Figure  3.9  22.  Sample  Reach  Envelope 


3.9. 5. 5  Mockups 

Description:  Mockups  should  oe  constructed  as  a  significant  part  of  the 
development  of  the  man-machine  system.  They  should  be  con¬ 
sidered  as  tools  which  are  used  to  evaluate  the  system  de¬ 
sign  before  the  actual  manufacture  of  system  hardware. 
Mockups  are  of  two  basic  types:  static  and  functional  (or 
dynamic).  The  static  mockup  is  a  full  scale  model  of  an 
item  of  equipment  or  a  facility.  It  is  usually  made  of  in¬ 
expensive  materials  such  as  cardboard  with  a  foam  core  or 
plywood.  All  major  Internal  components  are  represented  as 
actual  small  items  of  hardware  or  by  cutouts  of  drawings  or 
photographs  of  the  internal  items.  The  external  dimensions 
of  the  mockup  are  usually  not  critical.  Internal  dimensions 
having  to  do  with  workspace  design,  displays,  and  controls 
should  be  reasonably  precise. 

Functional  mockups  can  operate  in  limited  simulation  of  the 
prototype  equipment.  A  functional  mockup  has  controls  and 
displays  that  actually  work  as  compared  to  the  nonoperating 
static  mockup.  The  number  and  type  of  operations  that  may 
be  provided  in  a  functional  mockup  covers  a  wide  range.  The 
more  complex  functional  mockups  are  little  different  from 
simulators. 

Procedure:  The  mockups  should  be  made  initially  with  the  easiest  to  use 

and  cheapest  material  possible.  Various  thickness  plastic 
foam  core  filled  cardboard  sheets  may  be  used  quite  easily 
with  a  hot  glue  gun  and  a  sharp  matte  knife  to  build 
consoles,  racks,  and  even  complete  cockpits.  Console  panel 
layout  drawings  may  be  simply  glued  to  the  foam  core 
cardboard  to  simulate  the  appropriately  located  displays  and 
controls.  Test  participants  or  evaluators  may  simulate  the 
observation  of  displays  or  actuation  cf  controls  by  simply 
touching  the  drawing  and  performing  the  appropriate  hand 
(foot)  motion.  As  the  system  design  progresses  and  mockup 
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tolerances  become  more  critical,  plywood  material  should  be 
used.  Plywood  is  both  more  rigid  and  durable,  although 
considerably  more  costly  in  terms  of  construction  costs. 

The  plywood  mockups  may  be  converted  from  a  static 
representation  of  the  system  to  a  dynamic  or  hot  mockup, 
also  referred  to  as  functional  mockups.  The  console  panel 
drawings  which  were  glued  to  the  plywood  may  be  replaced  by 
the  actual  displays  and  controls. 

Use/Validity:  Wiring,  cabling,  piping,  and  ducting  may  be  designed  to 
visualize  three-dimensional  problems  from  scaled  down, 
two-dimensional  drawings.  Measurement  of  operator/ 
maintainer  subject  reach  capabilities,  clearance  spaces, 
access  opening,  and  vision  envelopes  can  be  determined  and 
compared  with  the  system  design  requirements  for 
verification.  Photographs  and  motion  pictures  may  be  made 
to  provide  coordination  aids  and  maintain  records. 

It  is  cheaper  to  develop  a  static  mockup  or  even  a  function¬ 
al  hot  mockup,  which  includes  the  proposed  electrical 
wiring,  than  it  is  to  build  prototype  hardware  with  numerous 
design  errors.  A  functional  mockup  makes  It  possible  to 
study  the  performance  of  personnel  in  simulated  operational 
situations.  The  HE  specialist  can  thereby  evaluate  opera¬ 
tional  characteristics  of  equipment  In  terms  of  human 
performance.  More  realistic  lighting  and  sound  measurements 
may  be  taken.  Procedures  may  be  verified.  Test 
participants  may  be  observed  and  interviewed  with  a  much 
greater  degree  of  confidence  as  to  the  validity  of  their 
responses.  In  addition  to  all  of  the  above,  mockups  along 
with  photographs  and  movies  provide  an  aide  to  design  pre¬ 
sentation  reviews  and,  later  on,  to  training  system 
development. 
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3. 9. 5. 6  Models 

Occasionally,  when  the  fabrication  of  full  scale  mockups  of 
hardware  or  facilities  would  be  too  elaborate  or  expensive 
to  construct,  scale  models  are  used  In  their  place. 
Unfortunately,  the  use  of  scale  models  negates  much  of  the 
value  for  HE  because  of  the  lack  of  good  HE  evaluation  tools 
such  as  three-dimensional  scale  model  manikins.  Models  are 
more  easily  transported  and  stored  than  mockups.  Models  are 
useful  to  perform  some  logistics  analyses,  but  cannot  be 
well  used  to  perform,  for  example,  MIL-STD-1472  checklists 
(Ref.  Figure  3.9-20). 

3. 9. 5. 7  Manikins 

A  tool  useful  for  evaluation  of  engineering  drawings  and 
sketches  is  the  two-dimensional  articulated  plexiglas 
manikin.  A  set  of  these  manikins  may  be  obtained  or 
prepared  in  a  range  of  sizes  and  scales  for  use  by  nc  or 
project  design  groups.  They  are  usually  made  to  represent 
two-dimensional  anthropometric  aspects  of  humans  as  seen 
from  the  side.  For  maximum  flexibility,  a  large  number  of 
sizes,  shapes,  and  scales  which  correspond  with  engineering 
drawing  practices,  (e.g.,  1/10  and  1/4  scale)  will  be 
required. 

The  manikins  are  used  by  placing  them  in  the  workspace  posi¬ 
tions  indicated  on  the  drawings  and  articulating  the  figures 
to  various  reasonable  positions  to  check  for  conditions  of 
interference,  access,  or  reach  availability.  To  a  limited 
extent,  visual  envelopes  may  be  checked.  If  the  required 
percentile  population  of  users  is  known,  e.g.,  5th  through 
95th  percentile,  then  the  manikins  should  be  used  to  check 
to  determine  if  the  design  is  compatible  with  each  of  the 
anthropometric  parameters  represented  by  the  5th  and  95th 
percentile  manikins. 
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Because  the  manikins  are  made  of  clear  plastic.  It  is  easy 
to  see  the  amount  of  Interference  of  overlap  if  the 
manikin's  dimensions  exceed  the  space  provided  on  the  scaled 
drawing. 

Frequently,  the  manikins  may  be  used  by  engineers  or 
draftsmen  to  illustrate  a  drawing  '1th  sketches  of  various 
sized  personnel  in  various  critical  positions.  The  manikins 
are  used  as  a  template  around  which  the  engineer  or 
draftsman  would  draw  the  outline  of  the  properly  scaled 
person  In  the  desired  articulated  position  on  the  drawing. 

The  use  of  these  manikins  is  most  worthwhile  during  drawing 
preparation  and  evaluation.  Whereas  the  cost  of  the  manikin 
procurement  (In  terms  of  a  full  set  of  sizes  and  shapes)  Is 
several  hundred  dollars,  they  tend  to  save  this  expenditure 
by  the  proper  Initial  design  of  mockups  and  prototype 
hardware  rather  than  the  costly  redesign  of  the  same.  The 
manikins  do  have  limitations  In  that  they  cannot  possibly  be 
completely  and  properly  articulated.  As  with  any  type  of 
manikin,  they  represent  a  theoretical  person  and  they  are 
useful  for  determining  compliance  with  only  one  anthropomet¬ 
ric  parameter  at  a  time.  MI L— STD-1 472  requires  compliance 
with  ninety  percent  of  the  population.  Given  the 
population,  it  is  essentially  impossible  to  design  a  manikin 
or  manikins  which  guarantees  the  use  of  ninety  percent  of 
the  population.  To  compound  the  problem,  new  user  popula¬ 
tions  include  females.  This  makes  it  most  difficult  to 
define  what  the  combined  male-female  population  is.  The 
percentages  of  male  and  female  are  not  equal  and  the  shape 
of  the  blmodal  population  curve  is  undetermined  (Ref.  Sec¬ 
tion  3.9.6,  Statistical  Analysis).  The  manikins  are 
therefore  only  a  very  approximate  tool.  They  cannot  be  used 
by  themselves  to  determine  precise  design  compliance  or 
deviation  from  criteria. 
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Other  forms  of  manikins  have  been  developed  for  full  scale 
use  in  aircraft  escape  systems  and  other  similar  hazardous 
use.  Their  use  is  more  appropriate  to  the  test  and  evalua¬ 
tion  phase  of  HE  rather  than  the  design  phase. 

3. 9. 5. 8  Specifications 

One  of  the  most  important  methods  to  use  in  insuring  the 
adequacy  of  HE  design  in  the  system  is  to  include  applicable 
HE  design  criteria  in  the  hardware  specification.  Whereas 
the  overall  need  for  this  HE  task  is  presented  in  Paragraphs 
3.1.3  and  3.6.1,  it  is  the  job  of  the  HE  specialist  to  in¬ 
sure  that  applicable  HE  design  criteria  is  incorporated  into 
each  appropriate  hardware  item  specification.  Generally,  it 
is  easiest  and  safest  (in  spite  of  the  need  for  tailoring) 
to  call  out  all  of  MIL-STD-1472  as  a  requirement  for  each 
hardware  specification. 

All  major  hardware  Items  which  make  up  the  total  system 
require  individual  specifications.  In  accordance  with 
MII-ST0-490,  which  describes  how  to  prepare  a  specification. 
Paragraph  3.3.7  of  the  specification  should  be  used  to 
describe  the  requirements  for  human  performance  and  HE. 

3. 9. 5. 9  CAFES  Computer-Aided  Design  (CAD) 

One  of  the  HFE  analyst's  or  crewstation  designers'  jobs  is 
to  produce  crewstation  configurations  that  are  consistent 
with  mission  requirements,  constrained  by  military  design 
standards  and  specifications,  and  compatible  with  technical 
and  cost  considerations.  The  computer-aided  design 
submodel  enhances  the  analyst's  capability  to  integrate  all 
the  diverse  design  considerations  into  a  workable 
configuration. 
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An  overview  of  CAO  capabilities  is  illustrated  in  Figure 
3.9-23.  CAO  functions  include:  a)  geometry  description  for 
computer  storage/retrieval;  b)  proportionate  scaling  (expan- 
sion/contracting)  of  defined  crewstation  geometry;  c) 
customized  changes  (tailoring)  in  geometry  of  computer-stored 
configurations;  d)  interference  analysis  between  crewmember 
escape  and  a  specified  crewstation;  e)  vision  analysis;  f) 
reach  analysis;  and  g)  computer-generated  graphic  views  of 
crewstation  cross  sections. 

The  major  CAO  functions  are  classified  under  three 
categories: 

a)  Crewstation  Design  Development:  Provides  means 
for  computer  storage  of  crewstation  configurations 
by  scaling,  tailoring,  repositioning,  or 
rearranging  specified  subsystems. 

b)  Crewstation  Design  Analysis:  Computes  metrics  for 
reach,  vision,  and  escape  analysis  as  a  function 
of  configuration  design  and  crew  member  size. 

c)  Graphic  Functions:  Provides  graphical  output  of 
crewstation  designs  and  design  analysis  in  hard 
copy.  Provides  for  growth  to  include  interactive 
graphics  modes.  Graphic  data  outputs  can  include 
sectional  views,  perspectives,  and  production 
drawings. 

Table  3.9-2  indicates  the  applications  or  outputs  of  CAD.  References  39 
and  40  contain  a  more  complete  description  of  this  technique. 

CAFES  Crewstation  Geometry  Evaluation  (CGE) 

The  Crewstation  Geometry  Evaluation  program  was  an 
experimental  development  by  Boeing  and  JANAIR  to  establish  a 
standardized  method  for  evaluating  the  physical  geometry  of 
a  crewstation. 
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It  evaluates  the  physical  compatibility  of  a  seated  crewmem¬ 
ber  of  any  size  with  any  crewstation  based  on  a  specific 
crewstation  design.  Oata  on  the  crewstation  geometry  and 
the  sequence  of  tasks  to  be  performed  are  stored  in  a  data 
file.  Mathemati  routines  provide  dynamic  movement  for  a 
variable-sized  n.o  natical  man-model.  Numerical  indicators 
(hand/joint  travel),  physical  and  visual  interferences,  and 
reach  infeasibilities  are  output.  The  crewstation  compli¬ 
ance  with  certain  military  standards  and  specifications 
(e.g.  MI L -STD-1 333,  Ref.  45)  requirements  are  also  checked. 
The  general  process  is  depicted  in  Figure  3.9-24. 

CGE  is  a  highly  detailed  component  of  the  evaluation  portion 
of  the  CAFES  system.  It  takes  the  man-machine  function  al¬ 
location  results  of  the  FAM  (as  evaluated  by  workload  analy¬ 
sis  in  the  WAM),  a  detailed  crewstation  configuration  design 
(as  aided  by  the  CAO),  and  selected  crew  anthropometry  to 
evaluate  the  design  with  respect  to  potential  geometric  or 
physical  problems.  Anthropometric  data  reside  in  the  CGE 
data  file  and  crewstation  and  task  sequence  data  are 
appended  to  it.  Table  3.9-2  lists  the  applications  or 
outputs  of  CGE. 

3.9.5.10  Human  Engineering  Computer-Aided  Design  (HECAD) 

HECAD  is  an  Interactive  computer  graphic  program.  It 
consists  of  two  major  parts.  The  first  is  the  geometrical 
part  with  which  a  workspace  designer  can  ar. ange  control  and 
display  components  into  a  workspace  configuration.  In  the 
second  part,  a  number  of  analyses  are  available  for  evalua¬ 
ting  the  arrangement  cf  components  against  one  or  more  crew 
•ation  task  ?  ■*  nee.  (>ef.  46,  Topmiller,  1978). 
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The  program  requires  three  main  inputs:  a  set  of  data 
describing  the  components  (name,  size,  type,  activation  time 
and  activation  reliability,  ana,  optionally,  coordinate 
information);  one  or  more  panels  described  by  their  corner 
coordinates;  and  one  or  more  task  sequences.  This  feature 
allows  the  designer  to  break  an  entire  mission  up  into 
smaller  and  more  manageable  task  sequences,  such  as 
preflight,  takeoff,  and  landing.  In  the  simplest  form,  the 
task  sequence  is  a  listing  of  the  component  identifying 
numbers,  in  the  order  that  they  are  activated  or  visually 
scanned.  There  are  also  provisions  for  communication  time 
and  for  machine  time,  when  the  operator  acts  on  the  machine 
and  then  the  machine  requires  some  reaction  time  (e.g.,  warm 
up). 

The  program  presents,  on  a  CRT,  a  perspective  projection  of 
the  work  station  with  panels  represented  by  outlines  and 
components  represented  by  dots.  The  designer  can  select  the 
point  from  which  the  projection  is  taken  and  change  it  at 
will.  Everything  is  located  in  a  set  of  orthogonal 
coordinates  whose  point  Of  origin  can  be  located  anywhere  in 
three-dimensional  space. 

Several  analyses  are  available  in  the  second,  analytical 
part  of  the  program.  First,  there  is  a  basic  reach  analy¬ 
sis  which  determines  the  distance  between  a  component  and  a 
shoulder  reference  point.  The  second  available  analysis  is 
a  visual  presentation  of  fingertip  paths  during  a  task 
sequence.  This  analysis  can  be  used  to  Identify  undesirable 
parts  in  the  task  sequence,  such  as  unnecessarily  long 
excursions  or  frequent  reaches  back  and  forth.  Third,  there 
Is  the  task  analysis.  This  analysis  takes  the  indicated 
task  sequence  and  examines  the  list  of 
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components  used  In  the  sequence.  The  fourth  analysis  was 
developed  because  of  the  dissatisfaction  with  previous  rel  1- 
ability  computations.  The  main  difference  in  task  time 
calculations  in  this  analysis  is  that  the  original  program 
assumed  a  single,  straight-through  sequence  of  panel  opera¬ 
tions  (called  the  dominant  path)  for  accomplishing  the 
task.  This  analysis  Introduces  allowances  for  additions  or 
deviations  (called  branches)  from  the  dominant  path. 

Armed  with  the  knowledge  gained  from  all  four  of  these 
analyses,  the  designer  can  decide  what  changes  should  be 
made  in  the  configuration  of  the  workplace.  The  computer 
does  not  order  any  changes;  they  are  strictly  up  to  the 
designer.  The  designer  may  have  good  reasons  for  locating  a 
certain  component  in  a  certain  place  (the  artifical  horizon 
in  an  aircraft  would  be  an  example).  Of  course,  the 
designer  can  also  contemplate  changing  the  order  of  actions 
if  the  equipment  permits  this. 

With  HECAD,  a  designer  can  assemble  a  workspace,  execute 
various  tasks  on  it,  identify  its  potential  design  short¬ 
comings  and  correct  them,  so  that  a  prototype  design  is 
quite  well  polished  before  one  cries  simulation  runs  in  a 
mockup,  or  other  operator-i n-the-loop  simulation  which  is 
usually  a  time  consuming  and  expensive  process.  However, 
there  still  is  room  to  grow  and  to  add  some  more 
capabilities.  One  of  the  considerations  during  the  develop¬ 
ment  of  this  model  has  been  to  assemble  a  procedure  or  de¬ 
sign  tool  that  is  easy  to  understand  and  apply,  requires  a 
minimum  of  preparatory  work,  and  quickly  produces  meaningful 
results:  in  other  words,  a  technique  that  workspace 
designers  would  find  desirable  and  useful,  and  one  that  is 
well  human  engineered. 
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3.9.5.11  Computerized  Biomechanical  Man-Model  (COMBIMAN) 


The  Air  Force  Aerospace  Medical  Research  Laboratory  Is 
developing  a  computerlzec  biomechanical  man-model  called 
COMBIMAN.  This  on-line  interactive  computer  model  was 
conceived  as  a  three-dimenslonai  manikin  for  workplace  de¬ 
sign  and  evaluation.  COMBIMAN  has  important  applications  In 
the  evaluation  of  existing  workplaces,  design  of  new 
workplaces,  selection  of  criteria  for  personnel  to  fit 
workplaces,  and  mapping  visibility  plots  (Ref.  47,  Evans, 
1978). 

Because  a  worker  functions  in  three-dimensions,  it  is  diffi¬ 
cult  to  evaluate  adequately  a  workplace  from  a 
two-dimensional  drawing.  While  mockups  provide  a 
three-dimensional  representation,  construction  of  a  good  one 
is  both  time  consuming  and  costly.  The  mockup  evaluation  is 
also  limited,  because  it  Is  difficult  to  find  subject  opera¬ 
tors  who  adequately  represent  the  anthropometric  variability 
of  the  user  population,  a  limitation  which  has  led  to 
erroneous  conclusions.  A  mockup  requires  some  cost  and  ef¬ 
fort  to  modify.  Thus,  it  can  become  an  obstacle  to  design 
change. 

COMBIMAN  does  not  share  these  handicaps.  It  is  a  three- 
dimensional  model  which  may  be  moved  about  and  viewed  from 
any  angle.  Since  the  man-model  and  workplace  design  exist 
only  on  a  CRT  display  and  in  a  computer  memory,  there  Is  no 
significant  investment  of  time  or  materials  in  effecting 
modifications.  Because  the  user  can  modify  the  design 
easily  while  sitting  at  the  display,  the  resistance  to 
change  is  eliminated  and  experimentation  is  encouraged.  Al¬ 
ternative  designs  may  be  thoroughly  evaluated  and  then 
permanently  recorded  by  means  of  a  pictorial  plot  or  tabular 
printout  of  the  workplace  data  and  man-model. 
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The  variable  geometry  of  the  COMBIMAN  allows  the  user  to 
define  quickly  a  series  of  man-models  which  represent  the 
entire  anthropometric  range  of  a  given  user  population.  A 
variety  of  special  problems  can  be  evaluated  by  generating 
realistic  ranges  of  certain  body  segments,  while  proportion¬ 
ing  the  remaining  segments  to  achieve  a  reasonable 
configuration.  With  COMBIMAN,  the  operator  can  specify  a 
certain  sitting  eye  height  and  the  program  will  generate  a 
man-model  with  realistic  proportions.  The  user  is  prevented 
from  selecting  an  unrealistic  combination  of  body-segment 
dimensions  by  constraining  equations  which  are  derived  from 
the  actual  anthropometric  data  base  of  the  population  being 
considered. 

The  man-model  itself  is  constructed  in  three  stages.  The 
fi.st  stage  is  the  generation  of  the  link  system  consisting 
of  33  segments  which  correspond  functionally  to  the  human 
skeletal  system.  The  second  stage  is  the  definition  of  the 
enfleshment  ellipsoids  (a  three-dimensional  ellipse)  about 
the  link  system  joints.  The  third  stage  of  man-model 
construction  is  connection  of  the  ellipsoid  silhouettes  by 
tangent  lines. 

The  two  most  important  applications  of  COMBIMAN  are  in  (a) 
the  design  of  workplaces,  and  (b)  the  evaluation  of 
workplaces.  The  other  features  of  the  model  (variable 
anthropometry,  reach  envelopes,  visibility  plots,  etc.)  are 
used  in  support  of  these  two  primary  applications. 

The  COMBIMAN  is  a  valuable  and  powerful  tool  for  assisting 
the  engineer  in  the  design  cf  workplaces.  Starting  with  a 
list  of  requirements  fcr  a  workplace,  the  designer  can  call 
up  the  man-model  to  which  he  has  been  assigned  dimensions 
representative  of  cbe  population  of  intended  operators.  The 
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designer  can  then  quickly  define  the  various  control/display 
panels  around  the  man-model  indicating  the  cornerpoints  with 
the  lightpen.  These  are  then  connected  by  lines  to  indicate 
the  panels  which  are  not  only  created  on  the  display,  but 
are  also  entered  in  the  three-dimensional  storage  arrays  and 
can  be  printed  for  future  use.  The  designer  can  cause  the 
coordinates  of  a  point  to  be  displayed  simply  by  pointing 
the  lightpen  and  pressing  a  button.  The  displayed 
coordinates  are  in  inches,  full  scale  with  respect  to  a 
meaningful  reference  point  rather  than  in  arbitrary  units 
which  would  have  to  be  scaled  or  converted  in  order  to  be 
understood. 

Frequently,  the  area  available  for  the  workplace  is  prede¬ 
termined  or  at  least  constrained  by  some  maximum 
dimensions.  The  size  and  location  of  some  control  panels 
may  also  be  known.  If  workplace  constraints  are  known  in 
advance,  they  may  be  entered  from  one  or  any  combination  of 
these  input  devices: 

a)  Lightpen  (on  CRT) 

b)  Keyboard  (on  CRT) 

c)  Punched  cards 

d)  Magnetic  tape  storage 

e)  Disc  storage 

The  user  can  temporarily  prevent  certain  characteristics  of 
the  workplace  from  being  displayed,  without  removing  them 
from  the  workplace  storage  arrays.  To  eliminate  the  projec¬ 
tion  of  a  particular  control  panel,  the  user  simply  points 
the  lightpen  at  the  panel  and  presses  a  button.  This 
technique  allows  the  operator  to  unclutter  a  very  complex 
workplace.  After  the  workplace  has  been  designed  around  the 
man-model,  the  designer  may  evaluate  the  workplace  by  the 
following  method. 

i 

\ 
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A  major  feature  of  the  COMBIMAN  is  Its  utility  in  evaluating 
workplaces.  These  generally  fall  into  three  categories: 

a)  Existing  workplaces. 

b)  Conceptual  workplaces  (which  have  not  been 
constrv  v  J,  but  exist  as  an  engineering  drawing). 

c)  Workplaces  generated  with  the  lightpen  in  on-line 
de'ign  operations. 

Once  a  workplace  has  been  entered  into  the  program,  it 
exists  in  three  dimensions  and  can  be  made  to  interact  with 
the  man-model.  Although  the  CRT  is  a  two-dimensional 
display,  two  orthogonal  views  are  simultaneously  projected 
and  can  be  rotated  for  viewing  at  any  angle.  If  the  user 
wants  to  take  a  closer  look  at  some  feature  of  the  display, 
that  feature  can  be  magnified  to  the  desired  sizes.  Regard¬ 
less  of  the  scale  of  the  display,  all  coordinates  and 
dimensions  are  stored  in  full  scale. 

Presently,  the  operator  has  several  options  in  defining  the 
body  segment  dimensions  for  the  man-model: 

a)  Oirect  Measure:  Specific  individuals  are  entered 
into  the  model  from  the  keyboard  or  punched  cards. 
Although  this  method  is  rarely  used  in  designing 
workplaces,  it  is  very  useful  for  the  validation 
of  the  model,  which  is  in  progress. 

b)  Stored  Individual  Data:  Oata  from  anthropometric 
surveys  are  stored  on  computer  tapes.  Dimensions 
of  a  selected  individual  can  be  recalled  and  used 
to  dimension  the  man-model. 
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c)  Data  Base  Summary  Statistics:  Percentiles 
computed  from  large  samples  are  used  to  define  the 
man-model.  Because  a  5th  percentile  man  is  not  an 
assemblage  of  5th  percentile  body  segments,  the 
user  must  select  a  separate  percentile  value  for 
each  of  the  critical  variables  by  selecting  the 
desired  value  from  a  list  of  displayed  percentile 
values.  The  lightpen  is  used  to  check  off  the 
desired  percentile  value  as  each  critical 
dimension  is  successively  underlined. 

d)  Computer-Aided  Dimensioning:  Assists  the  user  in 
generating  abstract,  but  realistic  man-models  from 
anthropometric  survey  data. 

This  last  method  is  most  useful  for  workplace  evaluation. 

The  user  starts  by  defining  the  body  characteristic  most 
relevant  to  the  evaluation.  This  characteristic  may  be  a 
dimension  (such  as  sitting  height,  arm  length,  etc.)  or  a 
mass  (such  as  total  body  mass  or  some  segment  mass)  and  can 
be  defined  either  as  an  actual  measure  or  a  percentile 
value.  Of  all  the  methods  for  dimensioning  a  man-model  for 
workplace  evaluation,  this  one  is  the  most  useful.  It  is 
both  fast  and  accurate.  It  allows  the  user  to  call  up  a 
wide  range  of  man-models  with  critical  dimensions  determined 
by  the  nature  of  the  task. 

C0M8IMAN  can  define  a  complex  range  of  head  and  eye  posi¬ 
tions  with  great  accuracy.  Because  of  this  capability,  the 
incorporation  of  visibility  plots  into  the  COMBIMAN  programs 
was  a  logical  development.  In  addition,  because  of  the  ease 
and  accuracy  with  which  the  program  handles  three- 
dimensional  geometry,  the  COMB1MAH  visibility  plots  contain 
additional  information  which  increases  the  utility  of  the 
output,  specifically,  the  three-dimensional  coordinates  of 
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the  workplace  with  respect  to  the  viewing  angle.  Using  the 
cockpit  as  an  example,  the  visibility  plot  program  scans  the 
frame  of  the  canopy  and  plots  the  vertical  viewing  angle  for 
each  integer  degree  within  the  horizontal  f ield-of-view. 

The  printout  shows  the  three-dimensional  coordinates  of  the 
canopy  frame  at  each  five-degree  increment  of  the  horizontal 
angle.  These  coordinates  are  given  in  the  aircraft  coordi¬ 
nate  system,  so  that  any  point  in  question  may  be  precisely 
located  on  the  cockpit  drawing.  Such  a  correlation  between 
look-angle  and  workplace  coordinates  makes  this  type  of  vis¬ 
ibility  plot  extremely  useful  to  the  design  engineer  since 
it  provides  accurate  feedback  of  the  effect  of  hardware  mod¬ 
ifications  on  the  external  visibility  of  the  pilot.  When 
evaluating  the  external  visibility  characteristics  of  a  cer¬ 
tain  cockpit,  the  designer  can  easily  vary  any  of  the 
following: 

a)  Size  of  the  operator  (such  as  sitting  eye  height 
based  on  relevant  anthropometric  surveys). 

b)  Seat  adjustment  (vertically,  horizontally,  or 
both) . 

c)  Head  position  (which  may  be  a  complex  function  of 
upper  body  position). 

d)  Visual  restrictions  (helmet,  helmet-mounted 
displays,  etc.). 

3.9.5.12  Computer  Accommodated  Percentage  Evaluation  (CAPE) 

Aircraft  cockpits  and  many  other  workspaces  traditionally 
have  been  designed  without  knowledge  of  the  proportion  of 
the  user  population  that  is  accommodated  with  safety  and 
full  capability.  In  aircraft  cockpit  design,  for  example, 
designers  have  been  directed  to  develop  cockpits  that 
accommodated  5th  through  95th  percentile  operators. 

However,  crew  system  designers  are  designing  for  the  5th 
through  95th  percentile  population  only  one  dimension  at  a 
time. 
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The  combination  of  all  the  necessary  dimensions  that  make  up 
a  workspace  design,  limits  the  operators  to  a  much  smaller 
actual  range  than  5th  through  95th  percentile.  It  has  been 
shown  that  more  than  52  percent  of  the  1964  population  of 
naval  aviators  would  be  excluded  when  5th  and  95th 
percentile  critical  limits  are  imposed.  This  led  to  the  de¬ 
velopment  of  CAPE  which  is  a  Monte  Carlo  model  for 
generating  representative  pilot  anthropometric  features,  a 
link-man  model,  and  an  adjustable  workspace  model  so  that 
the  workspace  accommodated  percentage  could  be  estimated  and 
maximized  (Ref.  48,  Bittner,  1975). 

The  computerized  accommodated  percentage  evaluation  (CAPE) 
model  has  two  options:  exclusion  demonstration  and  cockpit 
analysis.  Each  option,  and  its  underlying  model  with 
components,  is  described  in  summary  form  below.  More  de¬ 
tailed  descriptions  of  model  options,  their  components  and 
the  total  CAPE  model  are  contained  in  Reference  48. 

An  exclusion  demonstration  determines  what  percentage  of  a 
potential  population  is  excluded  from  a  workspace  design 
with  respect  to  each  anthropometric  feature  entered  into  the 
program.  This  CAPE  option  may  be  considered  to  be  composed 
of  two  components,  an  exclusion  limits  component  and  a  Monte 
Carlo  sample  generator. 

The  Exclusion  Limits  provides  for  the  entry,  storage,  and 
utilization  of  user-provided  standard  score  limits  of  an¬ 
thropometric  variables  required  for  exclusion  studies.  For 
each  variable  involved  in  an  exclusion  demonstration 
analysis,  high  cutoff  and  low  cutoff  values  must  be  input  by 
the  user.  This  component  of  the  analysis  provides  for  the 
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sequential  testing,  element  by  element,  of  Monte  Carlo¬ 
generated  standard  score  vectors  to  determine  if  the  vectors 
are  within  the  limits  set  by  the  high  and  low  standard  score 
boundaries  (populations  of  standard  scores  have  means  of 
zero  and  standard  deviations  of  one.)  Rejection  of  any 
component  test  is  defined  as  nonaccommodation  of  that 
(sample  subject)  feature  vector. 

The  Monte  Carlo  Sample  Generator  Component  generates 
quasi-random  vectors  of  standard  scores  that  match  a 
user-provided  correlation  or  correlation  square-root 
matrix.  It  is  based  on  a  method,  which  generates  standard 
score  feature  vectors  with  a  given  correlation  matrix. 
Conformable  vectors  of  quasi-random  normal  variants 
generated  by  a  subroutine  are  premultiplied  by  the 
square-root  of  the  desired  correlation  matrix  to  produce  a 
quasi-random  score  vector.  This  vector  can  be  viewed  as  a 
sample  subject  feature  vector  whose  elements  have  been 
converted  into  standard  scores. 

The  cockpit  analysis  determines  the  percentage  of  a 
population  that  will  be  excluded  from  a  cockpit  design  based 
on  the  geometric  parameters  of  the  workspace.  The  cockpit 
analysis  option  of  the  CAPE  program  can  be  thought  of  as 
being  composed  of  four  components:  a)  a  pilot  link  system 
component,  b)  a  sample  pilot  generator  component,  c)  a 
component  characterizing  a  seat-cockpit  layout,  and  d)  a 
cockpit  testing  component. 
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3.9.6  Test  and  Evaluation 


In  order  to  verify  the  man-machine  aspects  of  system  design  and  to  gather 
data  for  use  in  design  of  later  systems,  a  concerted  HE  T&E  effort  must  be 
accomplished.  During  this  period  of  system  development,  the  human  engi¬ 
neer  has  several  Important  responsibilities: 

a)  Assurance  that  applicable  HE  T&E  requirements  are  accomplished; 

b)  Demonstrate  conformance  of  system,  equipment  and  facility  de¬ 
sign  criteria; 

c)  Confirm  compliance  with  performance  requirements  where  the  op¬ 
erator  or  maintainer  is  a  significant  part  of  such  system 
performance; 

d)  Obtain  quantitative  measures  of  system  performance  which  are  a 
function  of  man-machine  interaction;  and 

e)  Determine  if  undesirable  design  or  procedural  aspects  have  been 
introduced. 

Many  of  the  most  popular  T&E  techniques  which  are  appropriate  for  use  of 
HE  are  presented  in  the  following  sections.  Depending  on  the  nature  of 
the  program,  only  a  few  of  these  techniques  would  normally  be  used. 
Sufficient  time  or  HE  evaluator  effort  does  not  exist  to  use  anywhere  near 
all  of  the  techniques  for  a  single  program.  Table  3.9-8  is  provided  to 
compare  the  T&E  techniques  on  the  basis  of  their  time  to  perform,  complex¬ 
ity,  cost,  and  cost  effectiveness.  References  49  (Meister,  1965)  and  50 
(Potempa,  1968)  provide  additional  information  on  many  of  the  HE  T&E  tech¬ 
niques  included  herein.  References  50  and  51  (Geer,  1977)  also  provide 
data  on  additional  HE  T&E  techniques  not  included  herein. 

3.9.6. 1  Direct  Continuous  Observation 

Description:  This  technique  is  simply  the  process  of  taking  a  relatively 
continuous  record  of  the  task  or  work  activity  or  some 
aspect  of  the  test  performance.  The  operation  may  consist 
of  an  observer  keeping  a  running  log  or  description  of  the 
test  activity  as  he  understands  it.  The  data  may  be 
recorded  by  hand  or  on  a  clip  board,  or  some  of  the  more  so¬ 
phisticated 
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techniques/tools  may  be  used  for  recording  events  and 
times.  Automatic  recording  techniques  such  as  photographs, 
movies,  and  sound  and  video  tapes  may  also  be  used  along 
with  direct  observation. 

Procedure:  The  detail  of  the  observed  data  is  in  accordance  with  the 

basic  considerations  indicated  in  Section  3.9.  The  observer 
should  be  skilled  at  being  able  to  discriminate  what  signif¬ 
icant  events  occur  during  the  test.  These  events  should  be 
summarized  and  interpreted  for  later  action.  The  observer 
must  be  familiar  with  the  anticipated  man-macnine  system 
performance.  He  will  observe  test  participants  while  they 
are  using  either  mockups  or  actual  hardware.  The  observer 
should  be  particularly  interested  in  obtaining  data  on  oper¬ 
ator  task  times  and  errors.  Data  as  to  the  observer's 
estimates  of  participants'  training,  skills  and  quantities 
should  also  be  recorded.  Life  support,  safety  and  hardware 
design  criteria  may  also  be  observed. 

The  use  of  the  direct  observation  technique  involves  the  use 
of  mockups  or  nardware.  Therefore,  the  most  appropriate 
time  to  use  this  technique  would  be  any  time  after  the  sys¬ 
tem  concept  has  evolved  sufficiently  to  produce 
three-dimensional  mockups. 

Use/Validity:  Observation  is  one  of  the  most  common  methods  of  evaluating 
personnel  and  system  performance.  It  is  used  to  some  extent 
in  some  form  in  every  test  and  evaluation.  Despite  the  in¬ 
creasing  use  of  automatic  recording  devices,  the  requirement 
for  direct  observation  will  never  be  completely  eliminated. 
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Observation  may  be  used  on  any  portion  of  a  total  system,  a 
subsystem,  or  on  system  components.  It  is  useful  for  T&E  of 
single  task  performance  or  the  simultaneous  operation  of 
several  tasks  by  several  test  participants.  It  is  simple  to 
perform  in  comparison  with  other  T&E  techniques. 

During  the  conduct  of  the  test,  it  is  possible  for  the 
observer  to  do  more  than  simply  record  test  occurrences. 

The  observer  may  evaluate  test  data  for  possible 
recommendations  or  test  action  items.  If  direct  continuous 
observation  is  not  used,  there  is  a  risk  of  missing  an 
overall  impression  of  the  test  as  well  as  random  test  events 
or  details  that  would  otherwise  be  overlooked. 

One  of  the  disadvantages  of  using  this  technique  is  the  re¬ 
quirement  for  specialized  observers  for  each  of  the 
different  test  aspects  or  categories.  It  is  seldom  possible 
for  a  single  observer  to  learn  a  sufficient  amount  about  all 
system  aspects  to  perform  an  adequate  job  of  observing  all 
system  tests.  The  use  of  continuous  observation  implies 
some  periods  of  test  observation  that  are  not  productive  in 
terms  of  gathering  HE  T&E  data. 

3. 9. 6. 2  Direct  Sampled  Observation 

Description:  This  technique  Is  identical  to  the  previously  listed  one 
with  the  exception  of  the  amount  of  time  spent  by  the 
observer  observing  the  test.  The  particular  times  chosen  to 
perform  test  observation  should,  of  course,  be  those  which 
coincide  with  the  performance  of  critical  tasks.  The  deter¬ 
mination  of  anticipated  critical  tasks  should  be  made  on  the 
basis  of  the  program's  preceding  systems  analysis  effort. 
Random  sampling  for  T&E  data  may  be  performed  if  possible 
critical  tasks  have  not  been  predicted  by  analysis. 
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Use/Validity:  The  only  difference  In  the  use  or  validity  of  the  sampled 
observation  technique  as  compared  to  continuous  observation 
Is  In  cost  savings  and  the  risk  of  missing  significant  T&E 
data.  It  stands  to  reason  that  If  the  tests  are  not  observ¬ 
ed  continuously,  the  test  observers  may  be  used  to  perform 
other  HF£  T&E  tasks  on  other  tests  or  In  data  reduction  and 
evaluation  of  previously  conducted  tests.  The  number  of 
personnel  required  to  perform  HFE  T&E  may  be  cut  by  a  factor 
of  one  half  or  more.  The  disadvantage  of  the  sampling 
technique  is  In  running  the  risk  of  missing  Important  opera¬ 
tor  performance  data  or  other  Important  HFE  related  data. 

If  critical  tasks  cannot  be  predetermined,  test  sampling 
should  be  performed  with  relative  frequency.  All  basic 
categories  or  types  of  operator/equipment  tasks  should  be 
observed  several  times  In  order  to  prevent  skewed  data. 

3. 9. 6. 3  Specification  Compliance  Summary  Sheet 

Description:  This  Is  a  form  that  Is  used  to  verify  that  system  perfor¬ 
mance  Is  In  accordance  with  specified  HFE  requirements. 
Briefly,  the  total  process  of  verifying  HFE  specification 
compliance  Is:  first  to  decide  the  best  method  to  verify 
the  specification  requirement  (l.e.,  analysis, 
demonstration,  or  quantitative  data),  second  to  perform  the 
analysis/test  and  third  to  document  the  results.  In  any 
case,  reports  are  written  as  to  the  analysis  or  test 
results.  The  Specification  Compliance  Summary  Sheet  Is  a 
way  of  summarizing  this  compliance  or  lack  of  compliance. 

Procedure:  The  evaluator  needs  first  to  have  a  thorough  knowledge  of 

all  HFE  aspects  of  the  contract  statement  of  work  and  the 
accompanying  system  specifications.  In  particular,  he 
should  understand  the  specification  Section  4.0  requirements 
(quality  assurance/testing). 
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After  the  test,  demonstration,  or  analysis  has  been  perform¬ 
ed  and  reported,  the  summary  sheet  form  is  completed.  The 
form  contains  a  space  to  indicate  the  specification  number 
and  complete  section  being  verified.  Space  Is  provided  for 
a  summary  of  the  test/analysis  results.  Signature  blocks 
are  provided  for  persons  preparing  the  summary  sheets  and 
approving  the  verification  of  specification  performance. 

Use/Val idlty:  This  technique  is  used  by  only  a  few  HFE  TiE  organizations. 

However,  this  lack  of  use  is  not  an  indication  of  the  need 
for  this  type  of  evaluation.  The  contract  and  related  sys¬ 
tem  specifications  are  by  far  the  most  Important  program 
requirements.  This  technique  is  unique  in  that  it  zeroes  in 
on  these  Important  requirements,  rather  than  concerning 
Itself  with  TtE  of  Indirect  system  requirements. 

The  Specification  Compliance  Summary  Sheet  is  an  excellent 
way  to  verify  the  Section  4.0  specification  requirements. 

The  only  disadvantage  associated  with  the  use  of  this  form 
Is  in  the  large  amount  of  time  required  to  fill  it  out.  The 
effort  preceding  the  use  of  this  form  may  be  considerable 
but  that  effort  Is  a  part  of  the  already  existing  HFE  T&E 
program.  If  this  technique  is  not  used,  there  Is  a  risk 
that  some  Important  aspect  of  HFE  design  criteria  may  be 
overlooked  both  by  designers  and  by  test  observers. 

3. 9. 6. 4  Technical  Order  Functional  Evaluation 

Description:  As  its  title  would  Indicate,  this  technique  is  designed  to 
evaluate  technical  orders  or  publications  pertaining  to  the 
test.  The  technique  is  based  on  the  use  of  a  form  to  be 
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completed  by  the  test  observers  while  they  are  performing 
the i r  other  direct  observations  of  the  test.  The  technical 
publications  must  be  evaluated  as  to  their  usefulness  and 
adequacy  in  three  areas: 

a)  Job  Instructions 

b)  Training 

c)  Job  Performance  Aids 

Job  Instructions  tell  how  to  accomplish  a  task  by  providing 
the  step-by-step  procedures  along  with  the  necessary  illus¬ 
trative  drawings.  Most  technical  publications  which  require 
validation  or  verification  provide  support  for  training. 
There  are  three  major  types  of  job  performance 
aids  which  are  identified  as  follows: 

a)  Job  Guides  (including  inspection  guideline 
manuals).  These  guides  contain  instructions  for 
fixed-procedure  tasks  such  as  checkout, 
adjustment,  removal,  and  replacement. 

b)  Fully  Procedural ized  Trouble  Shooting  Aids  spell 
out  the  steps  to  follow  in  isolating  malfunctions 
to  a  replaceable  or  repairable  unit.  The  steps 
start  with  observable  symptoms  of  malfunction. 

c)  Troubleshooting  Decision  Aids  provide  diagrammatic 
and  supporting  textual  information  which  will  help 
the  technician  decide  what  steps  to  take  in 
isolating  malfunctions  to  a  replaceable  or 
repairable  unit. 

The  following  sample  evaluation  form  (Figure  3.9-25)  is 
structured  so  that  the  first  three  questions  require  two 
judgments:  one  dealing  with  the  category  of  the  section 
being  evaluated  and  the  other  as  to  the  adequacy.  The  two 
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£  3A 

TECHNICAL  ORDER  FUNCTIONAL  EVALUATION 
ASSESS  the  USABILITY  OF  THE  IDENTIFIED  PARAGRAPHS  FOR  INSTRUCTIONS.  TAAININO,  AND/OR 
JOB  PERFORMANCE  AIDS.  (SEE  INSTRUCTIONS  ON  REVERSE  SIDE) 

T.O.  NUMBER: _ TITLE: _ 

EVALUATOR:  DATE: 


PARAGRAPHS  OR  SECTIONS  EVALUATED:  (GIVE  number  and  SUBJECT) 


NOTE:  PARAGRAPHS  OR  SECTIONS  COVERED  SHOULD  BE  CONSECUTIVE  AND  GROUPED  SO  THAT 
ANSWERS  GIVEN  APPLY  TO  ALL  PARAGRAPHS  LISTED. 


NAME 


T.O.  VERIFICATION  PERSONNEL  REQUIREMENTS 
AFSC  NAME 


AFSC 


CONOITIQNS  AT  VERIFICATION:  (INCLUOE  EQUIPMENT  INVOLVED.  WHERE  PERFORMED.  ETC.) 


EVALUATION:  (USE  ADDITIONAL  SHEETS  FOR  COMMENTS) 


t.  OO  THE  PARAGRAPHS  CONSITUTE  JOB  INSTRUCTIONS?  YES _  NO _ 

IF  YES.  ARE  THEY  ADEQUATE? 

2.  SMOULO  THE  PARAGRAPHS  BE  USED  FOR  TRAINING?  YES _  NO _ 

IF  YES.  ARE  THEY  ADEQUATE? 

J.  OO  THEY  CONSITUTE  JOB  PERFORMANCE  AIDS?  TES _  NO _ 

IF  YES.  ARE  THEY  aOEQUATE? 

«.  are  THE  STEPS  IN  LOGICAL  SEQUENCE  AND  DO  the  ELIMINATE  back  TRACKING 
WHERE  POSSIBLE? 

DIO  THE  INOIVIOUALS  DEMONSTRATING  THE  OPERATION  EXPERIENCE 
ANY  DIFFICULTY  AS  EVIDENCED  BY  ERRORS,  TOO  MUCH  TIME,  OR  NEED 
FOR  ASSISTANCE? 

ARE  THE  FUNCTIONS  OESCRIBED  SUFFICIENTLY  NEW  OR  COMPLEX  AS  TO 
REQUIRE  TRAINING? 

IS  IT  NECESSARY  TO  PROVIDE  ADDITIONAL  BACKGROUND  OR  SUPPLEMENTARY 
INFORMATION  IN  ORDER  FOR  THE  USER  TO  UNDERSTAND  WHAT?,  HOW?. 

WHEN?.  V/HERE?.  ETC.? 


YES _ 

NO_ 

YES _ 

NO 

YES _ _ 

NO 

YES 

NO_ 

YES 

NO 

YES _ 

NO_ 

YES _ 

N°_ 

Figure  3.9-25:  Sample  Technical  Order  Functional  Evaluation  Form 
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questions  are  to  be  answered  by  the  test  evaluator/observer, 
as  well  as  the  test  participants.  The  remaining  questions 
{4  through  7)  deal  with  the  qualitative  characteristics  of 
the  T.O. 

Procedure:  Most  sections  of  the  form  are  self-explanatory,  however,  the 

following  sections  should  be  completed  as  indicated: 

Evaluator:  Identify  individual(s)  interviewed  or  those 

contributing  to  the  evaluation. 

Paragraphs  Evaluated:  List  only  those  paragraphs  for  which 
the  evaluation  applies.  In  some  cases,  this 
can  be  done  in  large  blocks.  There  will  be 
some  events  where  several  separate  forms  will 
have  to  be  completed. 

T.  0,  Verification  Personnel  Requirements:  When  verification 
is  performed,  the  names  and  rate  (rank)  as  well 
as  skill  code  of  the  participants  is  required. 

Prior  to  conducting  this  type  of  evaluation,  the 
observer  or  evaluator  must  have  a  knowledge  of 
the  technical  manual  he  is  to  evaluate.  He  must 
also  be  familiar  with  estimated  system  and 
operator/maintainer  performance.  The  total 
technical  order  functional  evaluation  process 
will  result  in  either  verification  of  the  tech¬ 
nical  data  or  revisions  or  recommendations  for 
new  technical  data.  These  revisions  will  lc  co¬ 
ordinated  with  the  publications  writers. 

Use/Validity:  Depending  on  the  scope  or  charter  of  the  HFE  T&E  effort, 

technical  order  evaluation  may  or  may  not  be  performed.  If 
it  is  performed  (by  HE  personnel),  it  may  be  accomplished  at 
any  time  with  the  evaluation  of  any  evolving  systems  (as 
opposed  to  future  or  existing  systems).  The  effort  required 
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to  perform  this  evaluation  is  relatively  low  and  it  is 
therefore  recommended  as  a  task  to  be  accomplished  by  HFE  or 
other  personnel.  Failure  to  perform  this  evaluation  can 
result  in  several  maintenance  and  operational  mistakes  that 
would  otherwise  have  been  avoided.  The  cost  to  perform  the 
evaluation  must  be  considered  to  be  relatively  low,  particu¬ 
larly  compared  to  the  potential  cost  of  the  mistakes. 

3. 9. 6. 5  Human  Factors  Test  and  Evaluation  Manual  (HFTEMAN) 

Description:  HFTEMAN  must  Pc  considereo  as  considerably  more  than  an  HE 
T&E  technique.  It  is  designed  to  assist  the  HF  engineer  in 
the  areas  of  test  plan  preparation,  test  conduct,  test  data 
evaluation  and  analysis,  and  test  report  preparation.  The 
HFTEMAN  consists  of  two  documents:  the  first  contains  de¬ 
tailed  HFE  test  data  and  the  second  is  a  guide  book 
supplement  that  contains  specific  HFE  design  criteria  (Ref. 
52,  Navy,  1976). 

Procedure:  The  procedure  of  using  HFTEMAN  may  be  considered  as  a  five 

step  process.  This  procedure  is  well  detailed  on  the  first 
few  pages  of  the  manual.  The  first  step  requires  that  test 
items  be  classified  as  to  vehicles,  weapons,  electronics, 
etc.  The  second  step  is  to  identify  both  the  user  functions 
and  tasks  related  to  this  type  of  equipment;  in  other  words, 
a  selection  is  made  of  what  to  evaluate  and  the  criteria  to 
be  used  in  the  evaluation  tests.  The  third  step  decides  what 
human  factor  considerations  and  what  item  components  are 
relevant.  The  test  observer  should  review  the  task  list  and 
test  item  design  description  to  identify  which  of  the  test 
item  components  presented  in  the  matrix  apply  to  the  item 
under  test,  and  which  human  factors  considerations  are 
important.  In  the  fourth  step,  the  test  evaluator  goes  from 
the  cells  of  HF  considerations/task  item  components  to  cells 
containing  the  exact  test  criteria  as  indicated  on  a  separate 
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(opposite)  page.  The  last  step  is  to  prepare  the  HFE  test 
plan  which  includes  an  "objective"  (taken  from  HFTEMAN), 
"criteria"  (taken  from  HFTEMAN),  and  "methodology"  (taken 
from  the  HFTEMAN  Supplement).  The  "data  required"  also  are 
provided  in  both  the  HFTEMAN  and  HFTEMAN  Supplement. 

It  is  recommended  that  the  test  observer  be  thoroughly 
familiar  with  the  HFTEMAN  contents  before  he  starts  this 
procedure.  The  end  products  of  this  effort  should  be  both  an 
itemized  listing  of  all  HFE  system  deficient  items  and  a  gen¬ 
eral  feeling  of  pilot  or  other  operator  acceptance  of  the 
hardware  item. 

Use/Validity:  HFTEMAN  may  be  used  on  any  program  at  any  time  during  the 

program  evolution.  HFTEMAN  is  of  more  than  normal  value  in 
that  it  provides  both  the  basis  on  which  to  build  an  HE 
checklist  (Ref.  Section  3.9.5)  and  all  of  the  rest  of  the 
necessary  HFE  T&E  planning  and  conduct. 

HFTEMAN  has  broad  applicability.  No  special  test  equipment 
is  required  to  use  with  this  technique  and  it  will  be  of  use 
with  any  military  system.  If  HFTEMAN  is  not  used,  the  appro¬ 
priate  HE  test  planning  must  be  based  on  other  less  coordi¬ 
nated  resources. 

HFTEMAN  has  derived  from  the  U.S.  Army  TECOM  Human  Factors 
Engineering  Oata  Guide  for  Evaluation  (HEDGE).  The  Army 
guide  has  been  used  successfully  since  its  pubication  in 
1974.  Reference  53  (Army,  1074)  contains  additional  informa¬ 
tion  on  HEDGE. 
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3. 9. 6. 6  Environment  and  Performance  Measuring  Equipment 

There  are  several  different  items  of  test  or  measuring  equip¬ 
ment  that  are  extremely  useful  to  the  HE  test  observer.  A 
few  of  these  T&E  tools  are  presented  in  separate  sections, 
but  most  are  included  here.  The  following  subparagraphs  in¬ 
dicate  the  item  of  HE  test  equipment  along  with  a  brief 
description  of  its  use: 

a)  Photometer.  Measures  ambient  illumination  over  a  range 
of  levels  from  approximately  .005  to  25,000  foot 
candles.  This  is  an  extremely  useful  tool.  It  is  par¬ 
ticularly  valuable  for  verifying  specification  compliance 
with  light  level  requirements.  Sophisticated  mockups  or 
prototype  eauipment/facilities  are  required  for  the 
proper  use.  Most  photometers  are  relatively  easy 

to  use. 

b)  Spot  Brightness  Meter.  Measures  small  area  brightnesses 
in  foot-lamberts  within  angles  of  approximately  one 
degree  or  less.  This  tool  is  most  useful  tor  measuring 
prototype  hardware  display  brightness  such  as  from  LED's 
or  CRT's.  Specification  compliance  may  be  verified  with 
the  spot  brightness  meter. 

o)  Sound  Level  Meter  and  Analyzer.  Measures  steady  state 
sound  in  the  approximate  range  from  10  to  150  dB  for 
standard  weighted  noise  curves.  The  analyzer  provides 
active  band  analysis  for  the  more  critical  speech  range 
center  frequencies.  Specification  compliance  in  terms  of 
noise  curves  and  speech  interference  levels  may  be 
verified  with  this  equipment.  Hazards  to  test  personnel 
may  be  checked  prior  to  overexposure  conditions.  Most 
sound  level  meters  are  relatively  easy  to  use. 
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d)  Vibration  Meter  and  Analyzer.  Measures  amplitude  and 
frequency  components  of  complex  vibrations.  The  analyzer 
may  be  used  to  determine  amplitudes  at  selectable  fre- 
quency  bands  in  a  range  from  2.5  Hz  to  25  KHz.  Potential 
vibration  hazards  to  test  participants  may  be  checked 
before  actual  test  exposure.  Specification 
compliance  may  also  be  verified. 

e)  Thermometer.  Measures  air,  surface,  or  liquid 
temperatures.  May  provide  a  digital  readout  in  either 
Celsius  (centegrade)  or  Fahrenheit.  Should  include  capa¬ 
bility  for  attachment  to  several  temperature  sensor 
probes. 

f)  Anemometer .  *  'Sures  local  air  flow  in  the  range  of  0  to 
1000  ft/minute.  This  device  is  most  useful  for 
determining  crew  comfort  conditions. 

g)  Hygrometer  or  Psychrometer.  Measures  relative  humidity 
using  the  wet  and  dry  bulb  thermometer  method.  This 
device  is  also  very  useful  for  determining  conditions  for 
crew  comfort . 

h)  Gas  Tester.  Permits  convenient  short-term  sampling  and 
evaluation  of  many  toxic  gases,  vapors  and  fumes. 

i)  Force,  Torque  and  Dimension  Kit.  Various  instruments  for 
measurement  of  a  wide  variety  of  operator  or  equipment 
forces,  torques  and  distances.  The  force  measurement 
limits  should  be  from  1/4  oz.  to  250  lbs.  Torque  mea¬ 
surement  should  range  from  1/2  in.  -  lb.  to  160  fr.  - 
lbs.  A  tape  measure  should  be  capable  of  measuring 
distances  up  to  50  feet.  Scales  should  also  be  for 
measuring  centimeters,  millimeters,  inches  and  fractions 
of  inches.  A  protractor  is  useful  for  angular 
measurement . 
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j)  Anthropometry  Instrument  Kit.  Allows  measurement  of  sig¬ 
nificant  body  dimensions  using  the  anthropometer, 
spreading  calipers,  sliding  caliper,  gonimeter  and  tape 
measure.  The  measurement  of  test  participants  is  criti¬ 
cal  to  the  evaluation  of  workspace  layouts,  particularly 
when  egress  and  ingress  are  important  considerations. 

Care  should  be  taken  to  insure  the  proper  measurement 
procedures  are  adhered  to  while  obtaining  participant  an¬ 
thropometric  data. 

3. 9. 6. 7  System  Records  Review 

Description:  There  are  a  number  of  typical  test  and  evaluation  program 
records  that  may  be  useful  for  review  by  the  HE  personnel. 
This  technique,  the  review  of  system  T&E  records,  is  unique 
in  that  there  is  no  direct  contact  between  the  test  evaluator 
and  the  test  participants.  All  that  is  required  on  the  part 
of  HFE  evaluators  is  to  obtain  permission  to  review  the 
existing  test  records  and  to  go  ahead  with  the  tedious  task 
of  looking  through  them.  The  evaluator  should,  of  course, 
have  some  sort  of  system  knowledge  to  know  what  he  is  lookinq 
for  in  terms  of  anticipated  human  performance.  Typically, 
system  records  will  contain  test  logs,  maintenance  records, 
and  debriefing  records. 

The  HE  evaluator  may  find  data  on  equipment  operation 
problems,  technical  publication  inadequacies,  human  initiated 
errors,  and  training  inadequancies. 

Use/Validity:  This  technique  is  best  used  for  gathering  man-machine  perfor¬ 
mance  data.  Because  the  HE  evaluator  does  not  actually 
observe  the  test,  it  is  doubtful  that  sufficient  evaluation 
can  reliably  take  place  just  by  reading  a  word  description  of 
wnat  occurred.  Human  performance  tests  may  have  to  be 
scheduled  for  the  purpose  of  formal  observation  of  HE 
personnel . 
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The  problem  with  a  review  of  test  records  is  that  they  teno 
not  to  be  designed  for  gathering  human  factor.,  data.  What 
the  evaluator  is  able  to  obtain  from  these  records  may  De 
misleading.  There  is  significant  risk  that  HE  problems  that 
could  oe  readily  apparent  by  direct  observation,  are 
unobserved  or  obscured  by  other  less  significant  test  data. 

In  order  to  enhance  the  value  of  system  records  review,  the 
personnel  who  initiate  these  records  should  he  indoctrinated 
in  the  value  of  HE  and  HE  T&E. 

It  is  generally  agreed  that  the  use  of  this  technique  is  not 
required.  It  is  recommended  that  it  be  performed  only  when 
direct  HE  observation  is  not  possible.  The  debriefing 
records  should  be  the  most  useful  of  all  the  system  records 
normally  available. 

3. 9. 6. a  Test  Participant  History  Record 

Description:  This  is  not  a  direct  test  technique  dui  rather  a  method  of 
improving  the  test  evaluation  process.  The  Test  Participant 
History  Record  form  is  used  to  collect  data  on  personnel 
participation  in  the  tests,  if  possible.  Otherwise,  the  form 
may  be  completed  as  part  of  the  post-test  interview.  The 
sample  form  included  in  the  following  paqes  (Figure  3.9-26) 
emphasizes  participant  training,  experience  in  systems 
similar  to  the  one  being  tested,  and  participation  in 
previous  testing  related  to  the  same  over  all  system 
presently  being  tested.  This  form  may  need  to  be  modified  to 
suit  the  needs  of  the  particular  test  situation. 


PS  TEST  PARTICIPANT  HISTORY  RECORD 


Use/Val idity :  The  purpose  or  use  of  this  form  is  to  assist  in  the  evalua¬ 
tion  of  the  obtained  test  data.  For  example,  if  the  test 
participant  has  had  little  or  no  experience  ir.  performing 
tasks  similar  to  the  ones  he  has  been  given  to  do  as  a  test 
participant,  and  he  does  very  well,  then  the  conclusion  is 
that  the  man-machine  interface  being  tested  has  been  well 
designed  and  developed.  If,  on  the  other  hand,  his  perfor¬ 
mance  is  poor,  the  problem  may  or  may  not  be  due  to  poor 
man-machine  interface  desiqn.  A  more  experienced  test 
participant  will  have  to  be  given  the  same  tasks  to  perform. 
The  time  and  effort  it  takes  to  complete  the  form  is  small, 
and  the  potential  value  of  having  the  test  participant’s  sig¬ 
nificant  history  is  large. 

3. 9. 6. 9  Interviews 

Description:  Tne  HE  T&E  interview  technique  is  simply  the  process  of  the 
HE  test  evaluator  discussing  the  test  events  with  the  test 
participants.  Tnis  discussion  should  oe  structured  in  order 
to  insure  that  the  most  information  is  obtained  in  the  least 
amount  of  time. 

Specific  variations  to  the  general  interview  technique  may  be 
of  use  for  particular  situations.  For  example,  considerable 
test  and  evaluation  data  may  be  obtained  from  training 
instructors.  They  are  particularly  knowledgeable  in  reqard 
to  student  problems  with  new  systems  because  of  inadequacies 
in  the  system  desiqn. 

Procedure:  The  first  step  in  the  process  of  conducting  the  interview  is 

to  develop  a  format  for  asking  questions  of  the  participants 
(interviewees).  The  format  may  be  structured  like  a  check¬ 
list  to  insure  that  all  pertinent  aspects  of  the  test  are 
considered.  Tne  second  step  is  to  select  an  interviewer 
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who  has  had  experience  with  the  system  being  evaluated.  It 
Is  Important  that  he  has  observed  the  actual  test  conducted. 
The  next  step  Is  to  arrange  a  time  to  conduct  the  Interview 
with  the  test  participant. 

The  interviewee  should  be  questioned  about  the  task  he  has 
performed.  He  should  describe  what  he  thinks  his  test  task 
consists  of  In  terms  of  his  duties  and  those  of  others.  His 
opinions  should  be  obtained  on  the  adequacy  of  the  equipment, 
technical  data,  logistics  and  preparatory  training. 

The  Interview  should  be  conducted  as  soon  as  practical  after 
the  actual  test,  hopefully  within  a  few  hours.  If  possible, 
the  Interview  should  be  conducted  on  a  one  to  one  basis 
rather  than  one  Interviewer  questioning  several  participants 
at  one  time.  The  area  selected  for  the  Interview  should  be 
relatively  quiet  with  a  minimum  of  distractions.  The  time 
taken  to  conduct  the  interview  should  be  less  than  half  an 
hour.  Interviews  which  are  longer  than  this  start  to  get 
boring  and  become  an  Imposition  on  the  Interviewee. 

The  HE  Interviewer  must  take  care  to  insure  that  he  Is 
obtaining  the  Interviewees  actual  opinions  as  to  the  test 
situations  and  not  what  the  Interviewee  thinks  the 
Interviewer  wants  t.o  hear.  The  participant  must  be  assured 
that  he  is  not  being  graded  In  any  way  cn  his  responses.  The 
HE  Interviewer  should  try  to  quickly  develop  a  rapport  with 
participants.  If  the  participant  agrees,  a  tape  recording 
may  be  taken  of  the  Interview.  However,  whether  the 
participant  agrees  or  not,  some  Individuals  tend  to  be 
intimidated  by  the  use  of  tape  recordings  and  caution  must  be 
used  In  this  regard. 
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Another  example  of  an  Interview  technique  Is  the  "critical 
Incident  technique".  The  critical  Incident  technique 
consists  of  a  set  of  procedures  for  collecting  observations 
of  human  behavior  In  such  a  way  as  to  facilitate  their 
potential  usefulness  In  solving  practical  problems.  A  cr 1 t 1 - 
cal  Incident  Is  any  observable  human  activity,  the  purpose 
and  serious  effects  of  which  seems  clear  to  the  observer. 

The  five  step  procedure  Is  basically  at  follows:  a)  Determi¬ 
nation  of  the  general  purpose  of  the  activity;  b)  Development 
of  plans  for  collecting  Incidents  regarding  the  activity  and 
Instructions  to  the  persons  who  are  to  report  their 
observations;  c)  Collection  of  relative  objective  data;  d) 
Analysis  of  the  oata;  and  e)  Interpretation  and  reporting  of 
the  statement  of  the  requirements  of  the  activity.  The 
gathering  of  the  series  of  Incidents  consists  of  Inquiry  as 
to  most  effective  or  Ineffective  behavior  (or  critical 
Incident)  of  specified  activities/jobs.  Although  the 
Incidents  may  be  secured  by  interviews,  they  may  alto  ba 
obtained  by  written  responses. 

The  end  product  of  the  Interview  Is  a  quantity  of  test  data 
(facts  and  opinions)  to  review  and  evaluate  for  the  purpose 
of  presenting  system  problems  and  recommendations,  and  In 
many  cases  system  verification. 

Ute/Val idlty:  Tne  Interview  Is  one  of  the  most  significant  evaluation  meth¬ 
ods  used.  It  Is  a  simple,  low  cost,  quickly  used  technique, 
livery  test  Involves  a  certain  amount  of  test  data  that  cannot 
be  obtained  through  normal  observation,  Interviews  with  the 
test  participants  draw  directly  on  this  type  of  data  and  on 
the  knowledge  of  the  presently  available  system  expirts. 
Intorvlews  do  not  require  the  use  of  test  facilities,  They 
may  be  conducted  In  an  area  remote  from  the  test  site, 


The  purpose  of  an  interview  Is  to  find  out  either  objective 
facts  related  *c  the  system  about  which  the  Interviewee  has 
some  knowledge,  or  objective  facts,  attitudes,  or  opinions 
about  how  he  feels  about  some  test  aspect.  The  Interview 
must  be  designed  to  obtain  these  facts  with  as  much  clarity 
and  accuracy  as  possible. 

The  Interview  attains  Its  greatest  value  from  the  relation¬ 
ship  which  Is  established  between  the  Interviewer  and  the 
respondent.  In  a  properly  conducted  Interview,  where  a 
genuine  rapport  Is  established  between  the  Interviewer  and 
the  Interviewee,  It  Is  possible  to  obtain  more  detailed  and 
reliable  data  than  from  the  self-administered  questionnaire. 

One  caution  that  must  be  pointed  out  In  the  use  of  Interviews 
Is  bias  on  the  part  of  the  Interviewer  or  interviewee. 
Ideally,  the  Interview  results  In  the  Interviewee  supplying 
accurate  information  to  the  Interviewer,  However,  the  Influ¬ 
ence  of  bias  can  alter  the  results  to  such  an  extent  that  the 
answers  are  of  little  or  no  value  In  the  final  analysis.  The 
interviewer  may  bias  the  Interview  by  tone  of  voice,  the  way 
in  which  the  questions  are  phrased,  or  even  by  facial 
expressions,  These  and  other  sources  of  bias  con  be  greatly 
reduced  through  recog  ■  tlon  of  the  problem  and  by  training 
and  experience, 

Anoti  caution  associated  with  the  use  of  Interviews  Is  that 
they  cannot  be  used  at  a  substitute  for  direct  test 
observation,  They  should  be  used  as  one  of  several  HE  test 
and  evaluation  techniques.  Additional  data  on  Interview 
techniques  It  provided  In  Reference  54  (Army,  1975). 
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3.9.6.10  Questionnaires 
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Description:  The  basic  tool  for  obtaining  subjective  data  is  the 

questionnaire.  It  is  the  most  frequently  used  and  most  dif¬ 
ficult  to  construct  of  the  subjective  techniques.  The  ques¬ 
tionnaire  provides  a  structured  method  for  asking  a  series  of 
predetermined  questions  in  order  to  obtain  measurable 
expressions  of  attitudes,  preferences  and  opinions.  The  de¬ 
sign  of  a  questionnaire  which  will  produce  valid  and  reliable 
results  requires  a  measure  of  skill  and  experience. 
Unfortunately,  questionnaire  design  and  construction  cannot 
be  taught  from  books;  the  requirements  for  each  test  are 
somewhat  different  and  present  new  and  different  problems. 
However,  there  are  certain  rules  and  principles  of  question¬ 
naire  design  and  administration  which,  when  followed, 
eliminate  some  of  the  more  commmon  pitfalls  which  result  in 
faulty  questions  and  invalid  results.  The  following 
material,  especially  the  references,  are  intended  to  provide 
quidance  for  planning,  designing  and  administering  the 
questionnaire. 

Procedure:  The  method  of  questionnaire  design  applicable  to  the  types  of 

tests  conducted  by  HE  T&E  personnel  may  be  divided  into  seven 
logical  steps: 

a)  Preliminary  planning. 

b)  Selection  of  the  question  form. 

c)  Wording  of  the  questions. 

d)  Formulating  the  questionnaire. 

e)  Pretesting. 

f)  Administering  the  questionnaire. 

q)  Quantification  and  analysis  of  questionnaire  data. 
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The  preparation  of  a  questionnaire  requires  great  care  and  a 
background  knowledge  of  the  system  to  be  tested.  Knowledge 
also  is  required  regarding  the  background  of  personnel  to 
whom  the  questionnaire  will  be  administered,  and  the  type  of 
analysis  which  ill  be  made  of  the  results.  Too  often  a 
questionnaire  is  prepared  with  insufficient  planning.  The 
problems  involved  and  the  weaknesses  in  the  design  are 
frequently  not  recognized  until  such  time  as  the  results  are 
Interpreted. 

There  are  four  basic  question  forms  that  may  be  used  in  a 
questionnaire: 

a)  The  open-end  or  free-answer. 

b)  The  dichotomous  or  two-way. 

c)  The  multiple  choice. 

d)  The  rating  scale. 

Each  form  has  its  merits  and  disadvantages  of  which  the  ques¬ 
tionnaire  designer  must  be  aware  and  must  weigh  carefully 
before  final  selection.  No  one  question  form  is  superior  to 
the  others  In  all  cases.  In  order  to  select  one  form  over 
another,  the  designer  must  be  aware  of  the  advantages  and 
disadvantages  of  each  and  choose  that  form  which  best  meets 
the  needs  of  the  particular  test  situation. 

The  most  important,  and  also  the  most  difficult,  aspect  of 
questionnaire  construction  is  the  wording  of  the  questions. 
Most  authorities  agree  that  faulty  or  Improper  wording  of 
questions  accounts  for  the  greatest  source  of  error  in  the 
questionnaire  technique.  Errors  and  distortions  in  the  final 


207 


data  are  often  caused  by  misunderstanding  and 
misinterpretation  of  questions  due  to  use  of  an  improper 
vocaDulary  level  and  ambiguous  phrasing.  In  addition  to 
selecting  the  question  forms  and  wording  the  questions,  it 
also  is  necessary  to  consider  such  factors  as  the  sequence  of 
the  questions  and  the  format  for  presentation  and  data 
collection.  A  check  must  be  made  of  all  questions  to  insure 
complete  and  accurate  coverage  of  all  data  required  by  the 
test  objectives  and  test  critical  issues. 

A  questionnaire  is  subject  to  many  variables  and  must  not  be 
assumed  to  be  perfected  until  it  has  been  subjected  to  trial 
use.  The  pretest  provides  an  opportunity  to  try  the  ques¬ 
tionnaire  out  on  a  small  sample  of  respondents.  The  results 
of  this  trial  may  then  be  used  to  make  revisions  and 
improvements  as  necessary  before  test  administration.  The 
pretest  is  the  final  and  validating  step  in  the  method  of 
questionnaire  construction. 

The  product  obtained  from  administration  of  the  questionnaire 
consists  of  subjective  words  or  phrases.  This  information 
may  be  quantified  and  converted  to  figures  or  numbers  that 
can  be  tabulated  and  analyzed.  The  end  product  of  the  ques¬ 
tionnaire  may  be  a  simple  frequency  distribution  of  responses 
to  each  question  summarized  in  terms  of  numbers,  proportions 
or  percentages.  The  data  may  be  further  summarized  to  in¬ 
clude  averages,  standard  deviations,  or  correlations.  The 
summaries  also  may  include  statistical  analyses  snowing  the 
statistical  significance  of  differences  or  correlations 
obtained.  These  quantified  data  must  then  be  tabulated  and 
analyzed.  The  results  usually  are  summarized  in  tabular  form 
for  inclusion  in  a  final  report. 
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When  compared  to  the  Interview,  there  are  several 
similarities  and  differences  with  the  questionnaire.  Both 
the  questionnaire  and  Interview  should  be  conducted  within  a 
few  hours  of  the  test  for  best  results.  Both  techniques  may 
be  conducted  away  from  the  test  area.  Although  the  question* 
naire  must  be  more  structured  than  the  Interview,  the  ques¬ 
tionnaire  may  still  include  open-ended  questions.  The 
differences  are  In  that  HFE  personnel  need  not  be  present 
while  the  questionnaire  is  being  filled  out.  The  question¬ 
naire  is  inherently  easier  to  use  in  evaluation  or  analysis 
of  the  participant  responses. 

Use/Validity:  The  questionnaire  Is  a  subjective  measurement  tool  for 
systematically  obtaining  attitudlnal  responses  from  a 
selected  group  of  individuals.  The  function  of  the  question¬ 
naire  is  to  communicate  information.  When  properly 
formatted,  it  also  aids  in  the  tabulation  of  data  and  analy¬ 
sis  of  results.  The  questionnaire  is  used  to  assess  a  wide 
variety  of  qualitative  variables  such  as  acceptance,  ease  of 
use  and  preference.  It  may  be  administered  to  small  groups 
of  technical  personnel,  such  as  those  Involved  in  highly 
controlled  engineering  tests,  or  to  larger  representative 
cross-sections  of  service  personnel. 

Knowledge  of  individual  or  group  attitudes  provides  valuable 
information  regarding  reactions,  feelings,  and  preferences 
toward  military  systems.  Since  attitudes  determine  behavior, 
questionnaire  responses  of  a  representative  sample  o.  the 
population  permit  a  reliable  estimate  of  group  reactions  to 
systems  in  actual  use.  These  results  also  may  be  used  to 
anticipate  and  thereby  avoid  future  developmental  problems. 
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The  questionnaire  is  appropriate  for  use  in  all  types  of 
tests.  It  should  be  used  to  obtain  subjective  data  when 
objective  measurement  is  not  feasible  and  when  qualitative 
data  are  needed  to  supplement  objective  measurements. 

However,  it  should  not  be  used  in  place  of  direct  observation 
techniques  if  observation  is  possible. 

A  disadvantage  of  the  questionnaire  is  that  test  participants 
won't  respond  in  writing  to  the  degree  that  they  would  In 
talking  in  a  response  to  an  interview.  The  effort  to  write 
responses  to  open-ended  questions  is  greater  than  the  effort 
to  talk.  Another  disadvantage  of  the  questionnaire,  compared 
to  the  Interview,  is  the  inability  of  the  HE  observer  to 
pursue  a  participant  response  that  is  unexpected  but 
potentially  fruitful . 

One  of  the  most  difficult  problems  to  overcome  in  question¬ 
naire  design  is  the  misunderstanding  on  the  part  of 
individuals  as  to  what  a  questionnaire  is  and  how  it  should 
be  used.  There  are  those  who  believe  that  anyone  who  can 
write  well  and  use  a  little  common  sense  can  construct  a  good 
questionnaire.  The  seriousness  of  this  faulty  assumption  Is 
illustrated  by  the  fact  that  an  improperly  designed  and 
poorly  worded  questionnaire  will  still  yield  data  in  the  form 
of  numbers,  frequencies  and  percentages.  These  numbers  are 
amenable  to  statistical  analysis  and  may  even  produce 
statistically  significant  findings.  The  real  tragedy  Is  that 
these  erroneous  findings  may  be  used  to  draw  false 
conclusions  which,  in  turn,  contribute  to  faulty  critical  de¬ 
cisions  regarding  the  utility  of  an  item.  References  54 
(Army,  1975)  and  65  (Army  1976,  1979)  provide  additional  in¬ 
formation  on  the  use  of  questionnaires. 
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3.9.6.11  Motion  Pictures 


Description: 


This  technique  is  similar  to  the  use  of  video  tapes  (see  Par¬ 
agraph  3.9.6.13).  It  is  the  process  of  filming  participant 
performance  as  a  part  of  a  system  test. 

As  with  video  tapes,  actual  prototype  hardware  or  sophisti¬ 
cated  mockups  should  be  available  to  justify  the  use  of  this 
technique.  Less  sophisticated  mockups  imply  more  uncertainty 
in  design,  and  therefore  a  greater  risk  in  expending  a  motion 
picture  effort  on  unsuccessful  concepts. 

Trained  test  participants  must  be  available  for  observation 
of  their  appropriate  tasks.  The  cameraman,  and  particularly 
the  HFE  observer,  should  be  familiar  with  the  test  operation 
being  performed.  The  Knowledge  of  when  to  take  close-in 
footage  of  a  particular  critical  task  is  important.  As  in 
the  case  with  video  cameras,  a  dry  run  is  recommended  to  in¬ 
sure  the  filming  is  properly  performed.  Consultation  with 
all  personnel  familiar  with  the  anticipated  test  events  is 
advised. 

The  following  equipment  is  necessary  to  implement  this 
technique: 

a)  camera  and  (film) 

b)  lens 

c)  lights 

d)  projector 

e)  screen 

A  tripod  may  be  required,  depending  on  the  test  situation. 

Permission  to  use  cameras  in  secure  areas  must  be  obtained 
and  the  camera  equipment  and  cameraman  properly  scheduled. 
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Use/Validity:  This  technique  was  comparatively  more  useful  before  the  de¬ 
velopment  of  video  tapes.  Video  tapes  are  now  becoming  more 
popular  for  that  type  of  test  and  evaluation  process. 

However,  when  compared  to  all  other  techniques,  motion 
pictures  still  offer  the  advantages  of:  permanent  precise 
records  of  observed  events,  repeated  observations  of  the  same 
event,  slow  and  fast  motion  study  of  real-time  events,  use  in 
hazardous  areas,  and  record  of  task  activities  as  well  as  the 
related  background  situation.  The  data  gathered  may  be 
presented  to  large  groups  of  people. 

The  disadvantages  are  in  the  cost  and  effort  to  provide  the 
proper  equipment,  particularly  for  processing  and  viewing  the 
film.  Skilled  technicians  are  generally  required  for  the 
filming  of  motion  pictures. 

Motion  pictures  are  not  as  useful  as  video  tapes  in  that  they 
must  be  processed  to  be  viewed.  Instant  playback  of  a  film 
cannot  be  made  to  insure  the  adequacy  of  that  particular  test 
record.  After  the  processing,  a  projector  and  screen  are 
required.  The  film  cannot  be  reused  as  video  tape  can. 
However,  the  cost  of  the  least  expensive  movie  equipment  is 
less  than  the  least  expensive  video  equipment.  The  process 
of  recording  and  presenting  observed  test  tasks  in  slow 
motion  or  fast-action  is  cheaper  with  motion  pictures.  Re¬ 
ference  55  (Adams,  1962)  provides  more  information  on  the  use 
of  motion  pictures  for  HE  evaluation. 

3.9.6.12  Sound  Tapes 
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Description:  The  use  of  this  technique  is  now  so  common  that  a  description 
is  somewhat  suoerfluous.  Tape  recorders  are  now  both  inex¬ 
pensive  and  portable.  They  are  used  extensively  for  tasks 
other  than  formal  test  observation.  Their  use  in  HE  T&E  is 
somewhat  like  that  of  video  tapes  but  without  the  restric¬ 
tions  of  size,  security,  transportation  and  cost. 

Test  observers  commonly  use  sound  tape  recorders  to  maintain 
a  complete  record  of  test  conversation  and  events.  Test 
notes  may  be  verbally  entered  by  the  observers  themselves. 

The  recorders  may  also  be  used  to  record  participant 
interview  comments.  The  recorder  may  be  linked  into  the 
intercommunication  system  if  such  is  used  as  a  part  of  a 
large  scale  multioperator  test.  The  use  of  both  sound  tapes 
and  video  tapes  together  is  frequently  valuable. 

Use/Validity:  Sound  tapes  are  now  a  well  used  test/evaluation  technique. 

Their  use  is  extremely  easy  and  inexpensive.  They  have  the 
same  advantages  as  the  video  tapes  in  that  they  are  a 
permanent  record  of  events  (audio),  they  may  be  repeatedly 
reviewed,  they  may  be  used  with  time  tags  if  desired.  In  ad¬ 
dition  to  this,  sound  tape  recordings  negate  the  need  for  de¬ 
tailed  handwritten  notes. 

One  disadvantage  to  the  use  of  the  recordings  is  in  the 
quality  of  the  reproduction  if  a  high  ambient  noise  is  pre¬ 
sent  near  the  test  data  being  recorded.  Another  possible 
disadvantage  is  if  the  test  participant  becomes  self-con¬ 
scious  due  to  the  use  of  the  recorder.  This  would  be  more 
noticeable  during  an  interview. 

If  the  tape  recorders  are  not  used,  good  note  taking  becomes 
much  more  impo"tant. 
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3.9.6.13  Video  Tapes 

Description:  This  test  and  evaluation  technique  is  the  use  of  video 

cameras  and  related  equipment  to  make  video  tape  recordings 
for  detailed  review  and  evaluation  of  operator  and  mainte¬ 
nance  personnel  tasks. 

Actual  prototype  hardware  or  extremely  sophisticated  mockups 
should  be  available  to  justify  the  use  of  this  technique. 
Trained  test  participants  must  be  available  for  HE  evaluator 
observation  of  their  appropriate  tasks.  The  camera 
opera*  -(s)  and  particularly  the  HE  evaluator  coordinating 
the  data  recording  should  be  reasonably  familiar  with 

the  test  operation  being  performed.  The  knowledge  of  when  to 
use  the  zoom  lens  to  home  in  on  a  particular  critical  task  is 
important.  In  order  to  be  sure  all  the  more  critical  tasks 
are  properly  recorded,  dry  (or  test)  runs  of  the  test  may  be 
advisable.  Consultation  with  all  personnel  familiar  with  the 
anticipated  test  event  is  recommended. 

The  following  equipment  is  necessary  to  implement  this 
technique: 

1)  video  tape  recorder 

2)  camera  (preferably  portable) 

3)  zoom  lens 

4)  monitor 

5)  lights 

Additional  lenses,  monitors  and  tripods  may  be  desired  de¬ 
pending  on  the  complexity  of  the  test.  Sound  recording 
equipment  may  also  be  desired.  There  are  a  number  of 
easy-to-use  video  tape  recording  systems  which  might  be  made 
available  to  HE  personnel  at  the  test  sites  and  at  contractor 
f aci 1 ities . 
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Problems  associated  with  the  use  of  video  recordings  involve: 
the  logistics  of  transporting  the  equipment  to  the  test  site; 
the  security  of  the  equipment;  permission  to  record  any 
occurrences  in  secure  areas  (e.g.,  restricted  flight  line 
areas);  scheduling  of  the  video  equipment  and  a  cameraman; 
and  request  to  perform  recording  on  a  possible  test 
interference  basis. 

Use/Validity:  There  is  little  doubt  that  given  the  video  tapes  and  proper 
display  equipment,  the  use  of  this  technique  is  of  notable 
value.  However,  the  cost  effectiveness  of  the  technique  must 
be  considered  to  be  dependent  upon  the  complexity  of  the  task 
needing  evaluation.  Possible  transportation  and  lighting 
problems  should  he  considered  also  before  commitment  to  the 
use  of  this  technique. 

Careful  review  of  tape  playbacks  can  reveal  human  errors  and 
excessive  task  times  not  previously  capable  of  being 
detected.  The  application  of  maintenance  crew  teamwork  may 
be  examined.  Improper  procedures  may  be  thoroughly 
evaluated.  Improper  malfunction  determinations  may  be  traced 
back  to  the  point  of  the  original  mistake.  Technical  publi¬ 
cations  and  training  can  be  methodically  evaluated.  The 
adequacy  and  proper  use  of  tools  may  be  verified. 

Depending  on  how  they  are  used,  video  tapes  may  account  for 
less  test  interference  than  direct  test  observation  alone. 
This  would  be  true  for  an  equal  amount  of  test  data  gathered 
as  a  result  of  a  relatively  complex  test.  Once  recorded,  the 
data  record  is  permanent  and  may  be  presented  for  use  to 
numerous  persons  including  contractor  and  customer  alike. 

The  tapes  may  be  easily  stopped,  started  and  backtracked  for 
repeated  observation.  Each  task  may  be  thoroughly  examined 
step  by  step.  Test  sequences  that  may  not  be  properly 
recorded  may  be  easily  reviewed  and  retaken. 
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Further  advantages  include  the  fact  that  observer  errors  are 
reduced,  the  observation  can  be  recorded  and  observed 
remotely  from  what  might  be  a  hazardous  or  congested  area. 

The  tapes  may  have  considerable  use  as  training  aids.  They 
require  no  time  to  process,  but  motion  picture  films  do.  The 
tape  itself  is  reclaimable:  it  may  be  used  over  and  over 
again  for  different  tests.  The  record  of  time  tags  along 
with  the  video  is  possible. 

Disadvantages  of  the  technique  are  in  the  requirement  for 
special  personnel  or  training  required  to  use  the  recording 
equipment.  The  initial  cost  of  the  equipment  is  quite  high 
(several  thousand  dollars  for  the  recorder,  camera,  zoom 
lens,  monitor,  tripod  and  lights).  Slow  motion  and  stop  ac¬ 
tion  shots  are  possible  but  much  more  expensive.  If 
necessary,  the  one  alternative  technique  to  use  is  motion 
picture  film.  Additional  information  on  the  use  of  video 
tapes  is  provided  in  Reference  56  (Crites,  1969). 

3.9.6. 14  Photography 

Description:  This  technique  is  perhaps  too  simple  to  be  considered  as  such 
and  should  be  described  rather  as  a  HE  test  and  evaluation 
tool.  It  is,  very  simply,  the  process  of  taking  photographs 
of  whatever  tasks,  objects  or  events  that  are  pertinent  to 
the  HE  effort.  As  in  the  case  of  the  video  records,  actual 
prototype  hardware  or  mockups  must  be  available  to  justify 
the  use  of  the  tool.  HE  test  operators  must  be  familiar  with 
the  test  to  know  when  the  critical  tasks  or  events  require 
the  visual  record. 

In  addition  to  the  camera,  a  tripod  and  special  lighting  may 
be  required.  Flash  attachments  are  easily  used.  Depending 
on  facility  and  agency  requirements,  a  photographic  pass  may 
be  required.  The  location  of  the  test  may  restrict  the  us** 
of  cameras.  Polaroid  type  cameras  are  convenient  In  that 
they  provide  an  instant  picture  for  evaluation  as  to  the  need 
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for  additional  pictures.  However,  the  quality  of  the  Instant 
picture  cameras  tends  to  be  inferior  to  those  which  produce 
the  large  8  x  10  shots.  The  results  of  the  photography 
generally  are  appropriate  for  Inclusion  In  test  reports  or 
other  HE  test  and  evaluation  reporting  forms. 

Use/Val 1d1 ty:  Naturally,  photography  Is  a  well  used  HE  test  and  evaluation 
tool.  It  Is  easy  to  use  and  may  be  done  quickly.  The  par* 
tlcular  advantages  gained  In  using  this  technique  are  similar 
to  some  of  those  for  the  video  tapes  and  motion  pictures, 
e.g.,  the  photograph  Is  a  permanent  record  which  may  be 
reviewed,  It  may  be  used  as  a  training  aid,  and  decreases 
observer  errors  about  what  really  happened.  Photographs  are 
used  extensively  In  HE  testing  for  analysis  of  anthropometric 
Interface  problem;. 

The  obvious  disadvantage  associated  with  the  use  of  this  TIE 
tool  is  In  the  single  frame  static  picture  rather  than  the 
dynamic  picture  created  by  motion  pictures  or  video  tapes,  A 
small  problem  may  be  created  by  the  logistics  of  obtaining 
the  photographic  equipment  and/or  camera  personnel  and  tne 
permission  to  use  the  equipment  1r,  the  test  area.  Alterna¬ 
tives  to  photography  are  the  more  expensive  video  tapes  or 
motion  pictures  or  possibly  a  good  fast  sketcher  assigned  the 
duties  of  the  HE  test  observer,  in  a  few  Instances,  a  large 
number  of  descriptive  words  written  In  the  test  reports  may 
substitute  for  a  photograph  of  the  situation  or  equipment 
that  they  are  describing,  but  these  descriptions  era  seldom 
completely  satisfactory.  Reference  57  (Crltes,  1959) 
provides  more  Information  on  the  use  of  photography. 


3.9.6.15  Event  Recording- 

Description:  This  is  a  technique  or  method  for  recording  test  situation  or 
event  times.  The  equipment  involved  in  the  use  of  this 
technique  varies  in  complexity  from  the  stopwatch  to  complete 
systems.  The  more  complex  event  recorder  systems  might 
include:  an  event  recorder,  battery  pack,  event  control  box 
and  a  signal  cable.  The  event  recorder  itself  should  be  capa¬ 
ble  of  recording  on  several  channels;  the  battery  pack  is  to 
give  portability  to  the  operation;  the  control  box  is  used  to 
actuate  the  various  channels  in  the  recorder,  and  the  signal 
cable  is  to  electrically  tie  the  control  box  to  the 
recorder.  Other  recording  systems  are  provided  which  combine 
these  units  into  one  easily  portable  package. 

Procedure:  The  sequence  of  events  which  might  occur  with  the  use  of  this 

technique  may  be  as  follows:  HE  personnel  who  are  to  observe 
the  particular  test  first  become  familiar  with  the  planned 
test  events.  They  estimate  what  tasks  are  more  critical  and 
should  be  recorded  in  terms  of  time  performance.  If  the 
tasks  to  be  monitored  are  particularly  critical  they  may  even 
perform  a  dry  run  of  the  test  or  plan  to  run  multiple  repli¬ 
cations  of  the  time  critical  task.  The  total  test  may  be 
divided  into  several  functional  tasks  and  each  such 
assignment  allocated  to  a  separate  channel.  Examples  of  such 
task  functions  are  reading  technical  publications,  actuating 
controls,  reading  displays  and  making  adjustments.  The 
channel  controls  are  easily  activated  for  each  of  the  task 
functions  as  they  start  and  stop.  It  may  be  necessary  to 
write  start  labels  for  each  event  on  each  of  the  channels 
plotted  on  the  recorder  chart  paper  roll.  Figure  3.9-27 
shows  a  sample  of  this  type  of  annotated  record. 
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More  recently  available  recording  equipment  does  not  require 
the  use  of  the  paper  role  for  a  record  of  events.  The  test 
observer  simply  presses  combinations  of  keys  to  note  task 
functions  as  they  occur.  Data  entries  record  In  a  solid- 
state  memory  in  a  computer  program  format.  The  data  are 
later  transmitted  to  the  computer  by  connecting  the  device 
via  a  simple  connectlnq  cable.  In  this  manner-,  computer 
written  reports  may  be  written  in  minutes.  This  device  In¬ 
cludes  a  space  for  written  notes  on  an  Integral  note  pad. 

The  direct  outputs  of  each  of  these  event  recording  tech¬ 
niques  varies  from  handwritten  notes  to  complete  computer 
printouts  of  evaluate?-  data.  The  eventual  outputs  are  veri¬ 
fication  of  task  time  data. 

Use/Val Idlty:  Most  HE  test  and  evaluation  efforts  will  require  the  use  of 
one  of  the  following  (but  previously  indicated)  event 
recording  technique;  or  some  variation  thereof: 

a)  Event  recorder  and  separate  control  box 

b)  Combined  function  solid  state  memory  data  collector 
(OATAMYTE). 

c)  Stopwatch. 

When  critical  test  events  must  be  recorded  arid  evaluated, 
these  techniques  prove  valuable  for  determining  system/oper¬ 
ator  time  performance  capabilities.  Two  of  these  techniques 
allow  several  task  functions  to  be  recorded  at  once.  The 
observer  may  thereby  direct  more  of  his  attention  to  the 
other  aspects  of  the  test.  The  stopwatch  Is,  cf  course,  by 
far  the  cheapest  method  of  the  three  of  recording  time.  It 
may,  upon  occasion,  bo  the  most  cost  effective.  It  Is, 
however,  more  error  prone  than  the  other  methods.  The 
recordings  made  from  the  other  two  techniques  can  be  used  for 
timeline,  task  loading  and  time  sharing  analysis. 
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The  disadvantages  of  the  first  two  techniques,  when  compared 
to  the  stopwatch,  are:  the  cost,  requirement  for  a  test  with 
several  different  task  function  channels  occurring  simultan¬ 
eously  to  be  useful,  and  ease  of  use.  Technique  "b"  is 
better  than  “a"  in  that  it  is  easily  portable,  immediately 
compatible  with  existing  computer  programs,  and  includes  an 
earphone  timer  tone. 

In  general,  all  techniques  will  measure  objectively  human 
performance  and  provide  useful  data  for  the  test  as  a  whole. 
The  techniques  can  be  used  with  very  little  test 
interference.  The  training  required  to  use  the  technique 
equipment  varies  with  the  equipment  complexity  hut  is 
generally  uncomplicated.  The  data  are  applicable  for  time  to 
accomplish  tasks,  evaluation  and  optimization  of  tasks  in¬ 
volving  team  work,  and  the  isolation  of  specific  points  that 
degrade  turn-around  times,  loading  times  and  launch  times. 

The  technique  may  not  be  used  for  evaluation  per  se,  but 
further  analysis  must  be  made  of  the  data  using  other 
techniques.  Additional  information  on  the  use  of  event 
recorders  may  be  found  in  Reference  58  (Crites,  1969). 

3.9.6.16  Secondary  Task  Monitoring 

Description:  For  the  purpose  of  determining  crew  workload,  test 

participants  are  given  both  operational  tasks  and  secondary 
tasks  to  perform.  The  secondary  tasks  may  or  may  not  be 
meaningless  in  relation  to  the  rest  of  the  test  set  up.  They 
are,  however.  In  no  way  necessary  to  the  operational  tasks 
being  tested.  The  secondary  tasks  are  performed  with 
prototype  hardware  or  hot  mockups  on  special  equipment  that 
Is  Instrumented  through  hardwire  or  telemetry  to  record  crew 
performance. 


Procedure:  The  participant  Is  Instructed  to  perform  the  secondary  tasks 

when  not  required  to  perform  the  operational  tasks.  The  time 
taken  to  perform  the  secondary  tasks  Is  recorded  and 
subtracted  from  the  total  time  available.  In  this  manner*  the 
crew  workload  required  to  perform  the  operational  tasks  Is 
Implied  on  the  basis  of  the  measured  time  (or  effort)  not 
spent  doing  those  same  operational  tasks. 

Use/Validity:  This  is  a  useful  technique  to  measure  crew  workload  particu- 
larly  when  it  is  not  feasible  to  monitor  directly  the  opera¬ 
tional  performance  parameters.  Because  workload  can  be 
quantitatively  measured  In  this  case,  it  can  be  more  accurate 
than  many  other  workload  evaluation  techniques.  The  cost  and 
effort  to  implement  this  technique  is  relatively  high  as 
compared  to  several  other  HE  T&E  techniques  if  the  secondary 
task  data  are  recorded  automatically.  However,  the  cost  is 
inherently  lower  than  monitoring  operator  performance  on  each 
of  the  operational  controls  (and,  if  possible,  displays). 

There  are  two  basically  different  types  of  secondary  task 
monitoring.  The  first  type  uses  secondary  tasks  that  are 
completely  unrelated  to  the  system  operational  tasks.  These 
are  make-work  tasks.  The  second  type  is  more  sophisticated 
in  that  the  secondary  tasks  are  essentially  the  same  as  the 
required  operational  tasks.  Test  participants  seem  to  have 
more  motivation  to  do  the  more  real  secondary  tasks  rather 
than  the  make-work  tasks.  Reference  59  (Rolfe,  1971) 
provides  more  information  on  use  of  secondary  task 
monitoring. 
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3.9.6.17  Physiological  Instrumentation 


Description: 


Procedure: 


The  process  of  measuring  test  participant  physiological  data 
is  generally  quite  rigorous.  In  addition  to  all  of  the  set 
up  procedures  required  for  the  test  itself,  it  requires  sev¬ 
eral  important  tasks  that  must  be  performed  Just  for  the 
physiological  instrumentation. 

Physiological  measurement  requires  more  commitmment  from  the 
test  participants.  The  purpose  of  the  instrumentation  may  be 
to  monitor  physiological  parameters  to  insure  that  the 
participant  remains  in  a  safe  range  of  performance.  The 
implication  of  this  is  that  there  is  a  possible  unsafe  range 
of  performance  and  therefore  more  commitment  required  on  the 
part  of  the  test  participant.  Even  if  this  is  not  the  case, 
the  encumbrances  of  the  test  sensors  on  the  participant  are 
generally  somewhat  annoying. 

Trained  medical  personnel  must  approve  the  test.  Generally, 
they  should  perform  the  test  set  up  of  the  Instrumentation 
system.  This  would  involve  the  attachment  of  the  sensors  In 
a  manner  to  minimize  their  effect  on  the  total  test.  Medical 
personnel  must  also  be  present  during  the  test  if  any 
participant  risk  is  involved.  Electronics  technicians  may 
also  be  required  to  adjust  the  test  instruments. 

In  addition  to  the  individual  parameter  sensors  located  on 
the  participant,  wire  leads  must  be  provided.  Attached  to 
the  leads  would  be  the  appropriate  transmitters  (if  tele¬ 
metered),  receivers  and/or  amplifiers.  Instruments  for 
displaying  parameter  values  and  chart  recorders  will  also  be 
required. 
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Parameters  that  might  be  monitored  are  as  follows: 

a)  heart  rate,  blood  pressure 

b)  respiration  rate,  volume 

c)  galvanic  skin  response  (GSR) 

d)  electroencephalograph  (EE6) 

e)  electocardlograph  (EKG) 

f)  body  temperature 

g)  body  movement. 

Upon  completion  of  the  test,  medical  personnel  are  required 
for  analysis  and  evaluation  of  the  resulting  test 
physiological  data. 

Use/Val idity:  Physiological  measurement  is  performed  much  more  for  research 
testing  than  for  operational  or  field  type  testing.  It  is 
also  used  when  there  is  a  possibility  of  risk  involved,  for 
example,  centrifuge  runs.  Physiological  testing  is  seldom 
intended  to  measure  total  system  performance,  let  alone  the 
more  normally  monitored  operator  performance  parameters  of 
time  and  errors. 

The  cost  to  perform  this  type  of  testing  is  relatively  high 
and  the  effort  Involved  by  KFE,  medical  and  technical 
personnel  is  considerable.  Because  of  the  nature  of  the  test 
Itself,  which  would  require  the  use  of  physiological 
Instrumentation  for  safety,  the  testing  must  De  considered  to 
be  performed  on  an  interference  basis.  When  physiological 
monitoring  Is  really  needed,  there  is  no  substitute  technique 
that  may  be  used  to  obtain  the  necessary  data.  The  only  al¬ 
ternative  of  constantly  stopping  the  test  to  take  time  out 
for  the  reouired  measurements  is  unacceptable.  By  use  of 
radio  transmitters,  the  technique  may  be  monitored  remotely 
away  from  tha  test  area.  The  most  notable  use  of  this 
technique  has  been  in  manned  space  programs,  l.e.,  Skylab, 
Apollo,  Gemini  and  Mercury.  Reference  60  (Zonjer,  1971) 
provides  more  data  on  the  subject  of  physiological 

instrumentation. 
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3.9.6.18 


Physical  Measurement 


Description: 


This  technique  is  the  process  of  measuring  what  the  test 
participants  can  do  in  terms  of  their  physical  performance  or 
what  they  are  doing  in  terms  of  physical  and  cognitive 
performance.  Three  different  types  of  physical  measurement 
are  presented  in  this  section.  The  first,  anthropometry, 
deals  with  potential  test  participant  physical  performance. 
The  other  two,  oculometry  and  voice  monitoring,  pertain  to 
measurement  of  the  participants'  physical  and  cognitive 
processes. 

Anthropometry.  Anthropometric  measurements  may  be  made  of 
each  of  the  test  subjects  to  be  used  in  a  hardware  prototype 
or  mockup  test.  These  measurements  are  taken  on  the  assump¬ 
tion  that  the  test  will  indicate  various  areas  of  work  space 
or  work  access  verification.  If  problems  are  indicated, 
rather  than  designs  verified,  then  detailed  measurements  are 
taken  as  to  exactly  how  much  of  a  work  space  problem  exists. 
If  much  of  the  test  is  to  hinge  on  the  ability  of  the  Lest 
participants  to  fit  the  equipment  (e.g.,  cockpit  egress),  the 
subjects  may  be  specially  screened  and  chosen  to  fit  the 
worst  case  (larger)  population  percentiles  (95th  or  98th 
percentile).  If  a  subject  with  98th  percentile  buttock-knee 
length  the  98th  percentile  shoulder  breadth  can  successfully 
egress  with  the  given  cockpit  dimensions,  then  it  may  be 
assumed  that  most  pilots  will  be  able  to  do  the  same  at  least 
in  terms  of  egress  space. 

Oculometry.  This  is  the  technique  of  measuring  the  test 
participant's  eye  movement  while  he  is  seated  at  (in)  a 
mockup  or  prototype  hardware  of  the  system  being  tested.  The 
oculometer  is  used  to  view  the  participant's  eye  movement  in 
terms  of  deflection  rate  and  amount.  The  instrument  and  as¬ 
sociated  equipment  is  capable  of  recording  the  links  between 
controls  and  displays,  the  dwell  times  on  each,  the 
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Use/Validity: 


total  number  of  eye  contacts,  and  the  probability  of  next 
contact.  The  oculometer  performance  Is  at  a  half  degree  at 
30  inches  from  the  eye  within  an  envelope  30°  up,  10°  down, 
and  60°  horizontal.  Once  these  data  are  recorded,  panel 
layout  adequacy  is  verified  by  the  quantity,  location  and 
rate  of  eye  movements. 

Voice  Monitoring.  This  technique  is  performed  as  a  means  of 
psychological  stress  evaluation.  By  the  use  cf  sophisticated 
voice  monitoring  equipment,  similar  to  that  being  used  for 
lie  detection,  the  voice  is  analyzed  to  determine  stress. 

The  stress  indicates  test  situations  where  the  participant  is 
having  problems  or  is  close  to  the  point  of  having  problems. 
The  voice  stress  analysis  equipment  requires  operation  by 
trained  evaluators.  These  evaluators  should  be  familiar  with 
the  system  test  objectives  in  order  to  be  better  able  to 
analyze  test  data  and  to  recommend  problem  solutions. 

Physical  measurements  may  also  include  participant  muscular 
strength,  body  weight,  limb  coordination,  visual  and  auditory 
acuity,  and  kinesthetic  response. 

Anthropometry:  It  is  relatively  easy  to  measure  test 
participants  to  determine  their  anthropometric  measurements. 
The  fact  that  these  subjects  either  did  or  did  not  fit  the 
particular  mockup  or  prototype  is  also  easy  to  note  and 
record.  The  difficulty  in  the  use  of  this  technique  is  if 
and  when  particular  anthropometric  dimensions  are  required  as 
test  subjects.  It  is  very  difficult  for  HE  observers  to  go 
out  and  find  particular  anthropometric  dimensional  subjects, 
particularly  for  combinations  of  measurements  and  for  the 
extremes  of  the  population  (e.g.,  greater  than  90th 
percentile  and  less  than  10th  percentile). 
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The  real  value  in  using  anthropometric  measurements  Is  in  the 
knowledge  of  how  close  the  design,  as  represented  by  tne 
mockup  or  prototype,  comes  to  the  specified  user 
anthropometry.  The  disadvantage  Is  the  effort  in  finding 
subjects  who  properly  represent  the  required  population.  If 
this  technique  is  not  used  and  work  space  clearances  are 
critical  to  the  test  conduct,  the  HE  observer  runs  a  high 
risk  in  only  guessing  the  ant1  oometric  characteristics  of 
the  test  participants. 

Oculometry.  The  oculomete1"  technique  Is  relatively  complex 
and  expensive  to  use.  It  cannot  be  run  on  a  noninterference 
basis.  It  requires  trained  HfE  observers  to  use.  The  use  of 
the  technique  is  still  somewhat  experimental.  The  major 
advantage  in  the  use  of  the  technique  is  that  it  is  the  ideal 
way  to  perform  or  verify  cockpit  or  console  panel  link  analy¬ 
sis  data.  If  not  used,  questionaires  or  Interviews  may  be 
used  to  determine  subject  reaction  to  panel  layout  adequacy. 

Voice  Monitoring.  The  use  of  voice  monitoring  Is  both 
experimental  and  controversial.  Like  the  oculometer,  it  is  a 
complex  trchnique.  It  requires  trained  evaluators  and 
special  equipment  and  is  therefore  expensive.  Interpretation 
of  the  test  participant  voice  qualities  Is  variable.  On  the 
plus  side,  the  technique  may  reveal  problems  that  no  other 
technique  could  uncover.  The  only  alternatives  to  its  use 
are  Interviews  and  questionnaires  to  try  and  dig  out 
stressful  test  situations.  This  technique  has  been  used  In 
pilot  evaluation  during  aircraft  carrier  night  landings. 
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3.9.6. 1 9  Online  .'nteractive  Simulation 

Description:  Previous  HFE  T&E  technique  paragraphs  have  described  tech¬ 
niques  which  rely  heavily  on  prototype  hardware  or  mockups. 
Also  included  in  this  guide  are  several  techniques  which  do 
not  use  either  mockups  or  hardware,  but  are  instead  computer 
program  simulations  of  both  the  operator  and  equipment  in  the 
man-machine  interface  (e.g.,  CGE,  CAR,  CAPE).  The  general 
technique  described  in  this  section  pertains  to  the  use  of 
real  time  computer  program  simulations  and  actual  test 
participant  operators.  Like  other  simulations,  online  inter¬ 
active  programs  are  used  to  evaluate  and  demonstrate  the 
application  of  specific  procedures  and  equipment  to  specific 
operations.  It  is  often  difficult  to  make  a  sharp 
distinction  between  some  computer  simulation  set-ups  and 
functional  mockups.  The  emphasis  in  the  functional  mockup  is 
on  an  accurate  representation  of  spatial  dimensions  and 
arrangements. 

The  most  important,  requirement  of  an  online  interactive 
simulation  is  that  it  be  an  accurate  representation  of  some 
portion  of  the  proposed  system.  Critical  variables  in  the 
proposed  system  should  be  properly  duplicated  in  t.he 
simulation.  In  some  cases,  simulators  must  actually  provide 
deliberate  distortions  of  certain  parameters  in  order  to 
yield  operator  responses  that  will  be  valid  for  the  real 
world.  The  use  of  distortions  is  risky  but  often  necessary 
to  compensate  for  some  parameter  that  cannot  be  provided  for 
properly. 

Online  interactive  simulation  presumes  the  use  of  a  sophisti¬ 
cated  computer  and  software.  Test  participant  consoles  must 
also  be  provided  in  a  manner  similar  to  the  system  consoles 
being  simulated.  The  preparation  of  test 

228 


participant  operator  procedures  is  a  first  step  toward  the 
complex  job  of  constructing  the  real  time  interactive 
software.  Online  operation  requires  the  construction  of 
numerous  operator  commands  in  response  to  numerous  displays 
and  display  formats.  Operator  and  system  performance  outputs 
must  also  be  provided  for  in  terms  of  lists  and  time  plots  of 
events  versus  actions,  errors,  and  reaction  times. 

Use/Validity:  The  reason  for  using  online  simulation  is  because  of  the 
ability  to  find  out  what  might  occur:  to  manipulate,  to 
study,  and  to  measure  the  model  instead  of  the  real  world. 

There  are  several  advantages  to  using  online  simulation  as 
compared  to  other  methods  of  T&E: 

a)  Simulators  are  frequently  cheaper,  faster  and  easier 
to  construct  than  the  systems  or  prototype  hardware 
they  simulate. 

b)  Simulators  can  be  instrumented  to  collect  data  that 
would  be  difficult  or  impossible  to  octain  from  real 
systems  and  the  data  may  be  quickly  reduced  to 
usable  form. 

c)  Simulators  are  extremely  useful  as  training  aids. 

d)  Simulators  are  easier  to  manipulate  than  the  systems 
they  represent. 

e)  Simulators  may  be  used  to  perform  tasks  that  would 
otherwise  be  hazardous  to  the  test  participants 
(e.g.,  crashlandings) . 

f)  Once  the  simulation  program  has  been  provided,  al¬ 
ternative  procedures  or  tactics  may  be  easily 
manipulated. 

g)  A  record  of  data  may  be  kept  for  later  playback. 
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The  disadvantages  in  the  use  of  online  simulation  as  compared 
with  other  T&E  techniques  are  as  follows: 

a)  Simulation  tends  to  Invite  overgeneralization. 

b)  Simulations  may  be  wrong  because  of  incorrect  rela¬ 
tionships  that  have  been  made  to  hold  between 
variables,  or  assumed  constraints  may  be  in  error. 

c)  Simulators  may  add  ingredients  of  their  own  that 
will  not  be  found  in  the  real  world  system. 

d)  Simulators,  in  general,  are  very  expensive. 

The  time  to  use  online  simulation  is  generally  before  the 
construction  of  the  hardware  (and  software)  that  It  Is  to 
simulate.  If  this  is  not  done,  there  is  little  point  in  the 
expenditure  of  the  time  ar.d  effort  for  the  simulation. 

There  are  essentially  two  alternatives  to  the  use  of  online 
interactive  simulation.  One  simulation  technique  is  the  use 
of  man  model  programs  such  as  the  CGE,  CAR  and  CAPE  models 
previously  mentioned.  The  other  alternative  is  the  use  of 
all  the  T&E  techniques  which  utilize  the  direct  or  indirect 
data  obtained  from  the  actual  prototype  system  hardware. 

3.9.6.20  Statistical  Analysis 

Description:  This  section  on  statistical  analysis  techniques  is  applicable 
to  both  system  analysis  and  evaluation.  In  order  to  maintain 
consistency  between  this  section  and  other  HE  techniques 
sections,  the  details  of  the  numerous  statistical  methods 
cannot  possibly  be  provided  herein.  However,  a  few  of  the 
more  commonly  used  techniques  are  briefly  presented  along 
with  their  use.  These  techniques  have  been  grossly 
categorized  into  the  two  areas  of:  a)  statistical 
comparisons,  and  b)  user  population  selection. 
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Statistical  comparisons  may  deal  with  the  parametric  perfor¬ 
mance  of  two  or  more  hardware  items  under  consideration  for 
use  in  the  system  design.  Comparisons  may  also  be  made  be¬ 
tween  different  parameters  in  order  to  draw  a  conclusion  or 
develop  new  and  useful  data.  System  trade  studies  often  in¬ 
clude  performance  data  comparisons  such  as  reliability 
statistics.  The  mean  or  average  reliability  for  one  hardware 
item  being  considered  is  compared  to  another  hardware  item. 
Additional  factors  such  as  standard  deviations  from  the  mean 
and  item  population  are  necessary  to  make  a  proper  perfor¬ 
mance  comparison.  The  confidence  limit  or  level  of  the 
results  of  the  statistical  analysis  are  very  important. 

These  are  obtained  from  the  standard  errors  which  are,  in 
turn,  a  measure  of  the  sampling  uncertainty  (e.g.,  sample 
size).  Statistically  derived  data  are  of  little  value  with¬ 
out  an  associated  confidence  limit  (e.g.,  95%). 

User  population  selection  deals  with  the  selection  of  a 
sample  from  total  population.  It  is  generally  impossible  to 
test  or  measure  all  items  (or  users)  in  a  population  set  from 
which  data  is  to  be  obtained  and  analyzed.  Statistical  meth¬ 
ods  exist  for  random  or  specific  parameter  (i.e.,  stratified) 
population  sampling.  Whether  a  total  population  or  a  sample 
of  the  population,  the  data  obtained  will  be  presented  in 
distribution  plots.  These  plots  describe  the  frequency  of 
occurrence  of  the  individual  parameter  values  in  the  sample 
tested.  The  form  of  the  resulting  distribution  (e.g., 
Gaussian,  Poisson,  binomial)  is  important  in  selecting  the 
appropriate  statistical  techniques  to  be  employed  and  in  the 
conclusions  to  be  drawn  from  the  data.  For  example,  a 
bimodal  distribution  generally  indicates  that  the  data  sample 
was  actually  drawn  from  two  distinct  populations  and  the 


231 


Procedure: 


Use/Validity 


\ 

application  of  standard  statistical  techniques  may  not 
produce  the  intended  results.  As  a  further  illustration, 
recent  trends  in  design  criteria  application  require  the  com¬ 
bination  of  male  and  female  population  anthropometric  data. 
This  combination  will  produce  bimodal  distributions.  In  such 
situations,  standard  statistical  techniques  for  determining 
cost  effective  design  criteria  (e.g.,  choice  of  5th  through 
95th  percentile)  can  be  erroneous. 

It  is  not  the  intent  of  this  guide  to  provide  the  procedure 
for  each  of  the  many  statistical  analysis  techniques.  If  the 
HE  specialist  has  questions  concerning  data  analysis  and 
interpretation,  consultation  with  a  statistical  specialist 
should  be  employed.  This  consultation  should  occur  during 
the  early  planning  stages.  Errors  in  sample  selection  or 
data  collection  procedures  cannot  be  corrected  in  the 
analysis.  Statistical  analysis  that  once  was  performed  with 
the  use  of  desk  top  mechanical  calculators  is  now  quickly 
performed  by  computer/software  techniques.  If  possible, 
statistical  data  should  be  collected  in  machine-readable  form 
at  the  test  site.  At  a  minimum,  the  data  collection  format 
should  be  designed  for  ready  use  as  a  guide  for  key  punching 
of  input  cards. 

Although  HE  itself  is  a  specialized  field,  there  are  persons 
within  this  discipline  who  specialize  in  HE  statistical 
analysis.  The  majority  of  HE  personnel  have  little  to  do 
with  the  statistical  analysis,  both  because  of  relatively 
little  need  to  do  so  and  availability  of  a  few  well  qualified 
persons  who  can  perform  the  statistical  analysis  when  needed. 
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Comparisons  or  correlation  between  parametric  data  are  useful 
to  extrapolate  from  limited  data  bases.  For  example,  if 
based  on  comparisons  between  evaluator's  judgments  of  opera¬ 
tor  task  reliability  and  actual  empirical  data,  a  high  corre¬ 
lation  seems  to  be  evidenced,  then  this  correlation  can  be 
quantified  by  the  use  of  scatter  diagram  plots,  regression 
curves,  and  correlation  coefficients.  The  quantified  corre¬ 
lation  can  be  used,  with  some  caution,  to  extrapolate  to  op¬ 
erator  task  reliability  estimates  which  have  not  been  field 
tested.  Correlation  data  showing  the  relationship  between 
anthropometric  measurements  can  also  be  very  useful. 

Statistical  methods  are  not  used  as  often  as  they  should  be 
to  evaluate  parametric  data  used  to  perform  trade  studies. 
Often  hardware  selection  between  various  brands  and  systems 
is  made  on  the  basis  of  quoted  or  derived  performance  data 
that  is  not  statistically  reliable  (significant)  or  accurate. 

Just  as  statistics  can  be  of  great  value  to  the  HE  analysis 
and  evaluation  process,  it  can  also  cause  problems.  If  the 
statistical  analysis  is  attempted  by  persons  with  limited 
experience,  it  is  easy  to  make  mistakes  both  in  the  choice  of 
particular  statistical  techniques  and  in  the  application  of 
the  techniques.  At  the  same  time,  skilled  but  unscrupulous 
analysts  have  been  known  to  purposely  misuse  statistics  to 
"prove"  an  item  of  performance  data  which  does  not  actually 
hold  true.  A  thorough  analysis  should  be  made  of  any  data 
which  are  crucial  to  a  design  decision  and  which  could  be 
suspect. 
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3.9.7  Data  File 

The  contractor  HE  organization  shall  establish  and  maintain  all  HFE  and  HE 
related  data  generated  on  the  program  in  the  HE  Data  File.  These  data, 
such  as  the  HE  plan,  analyses,  design  review  results,  drawings,  checklists, 
and  other  supporting  background  documents  reflecting  HE  actions  and  deci¬ 
sion  rationale,  shall  be  maintained  and  made  available  to  the  procuring 
activity  at  the  contractor's  facility.  Typically,  these  data  will  be  re¬ 
viewed  at  various  contractor  meetings  such  as  design  reviews,  audits, 
demonstrations,  and  T&E  functions.  The  data  file  shall  be  organized  to 
provide  traceability  from  the  initial  identification  of  HE  requirements 
during  analysis  and/or  system  engineering  through  design  and  development  to 
the  verification  of  these  requirements  during  test  and  evaluation  of  the 
approved  design,  software  and  procedures. 

3.9.8  Baseline  Monitoring 

A  method  frequently  used  by  program  management  to  keep  both  the  program 
moving  and  the  design  improving  at  the  same  time  is  the  establishment  of  a 
baseline  configuration.  The  design  is  controlled  by  drawings  and  documents 
describing  the  total  system.  The  initial  configuration  is  established  by 
the  program  manager  with  the  assistance  of  the  chief  engineer  and  others. 
Prior  to  the  program  CDR  informed  contractor  meetings  are  called  to  review 
changes  to  the  baseline.  After  CDR  a  more  formal  change  board  is 
established  to  control  the  necessary  design  changes  and  their  accompanying 
documentation.  After  the  CDR,  the  baseline  is  bought-off  by  the  customer 
and  design  changes  must  be  approved  and  paid  for  by  the  Air  Force  (by  way 
of  Engineering  Change  Proposals:  ECP's). 

A  typical  baseline  configuration  might  start  out  during  the  conceptual 
phase  as  a  description  of  the  system  in  terms  of  required  system  perfor¬ 
mance  and  design  requirements.  This  will  eventually  evolve  into  configura¬ 
tion  item  performance  and  design  requirements  by  the  end  of  the  advanced 
development  (validation)  phase.  Configuration  item  product  definition  must 
be  maintained  through  the  full-scale  development  and  production  phases. 
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The  baseline  system  design  provides  a  single  source  for  all  program  groups 
to  quickly  reference.  This  is  most  necessary  in  order  to  make  quick  and 
accurate  trade  studies  to  determine  significance  of  cost  and  performance 
trade-offs.  The  baseline  configuration  provides  a  model  which  can  be  used 
for  planning  and  scheduling  purposes.  It  is  Imperative  that  manufacturing 
and  engineering  are  using  the  same  system  configuration.  It  is  imperative 
that  HE  personnel  monitor  the  baseline  configuration  to  be  sure  that  il  in¬ 
cludes  proper  consideration  of  the  man-machine  interface  and  necessary  HE 
design  criteria. 

3.10  Contractor  Monitoring 

After  the  conf-act  award  is  made,  contractor  monitoriig  can  be  accomplished 
in  a  number  of  ways.  These  are  the  HE  Program  Plan,  conferences,  design 
reviews,  trade  study  reports,  CORL  reports,  HE  data  file  review,  baseline 
configuration  review,  and  frequent  use  of  the  telephone. 

If  an  HE  program  plan  is  required,  it  must  be  reviewed  and  modified  if  nec¬ 
essary  within  a  few  weeks  from  the  start  of  the  contract.  A  program 
kick-off  meeting  for  just  HE  alone  is  a  good  ioea  to  discuss  any 
ambiguities  1  ri  the  plan  and  to  make  necessary  changes.  The  meeting  is  also 
helpful  In  that  the  customer  and  contractor  can  meet  face  to  face  and  go 
through  the  plan  section  by  section  prior  to  later  important  design 
reviews.  Tne  meetina  snould  be  at  the  contractor’s  facility  in  order  that 
t he  facility  Itself  and  the  work  (e.g.,  mockups)  already  performed  on  the 
contract  In  competition  can  be  shown  to  the  Air  Force  customer.  Once 
approved  bv  the  Air  Force  manager,  the  HF  Program  Plan  will  be  the  basis 
for  tne  HE  contractual  compliance. 

If  progress  reports  are  required,  they  must  be  re/iewed  and  evaluated.  The 
Air  Force  HE  responsibilities  in  reviewing  design  data  may  vary  from  com¬ 
plete  ri>spons1b11 1 ty  in  the  case  of  data  submitted  in  response  to  MIL  - H - 
AGdbS  or  ilE  CORL  items,  or  to  just  "comment"  or  concurrence  action  on  other 
data.  Tne  scope  and  purpose  of  the  review  is  to  assure  that  the 
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contractor's  efforts  are  of  acceptable  quality  and  in  accordance  with  the 
contract  specification  and  work  statement.  The  Air  Force  HE  manager  must 
also  attend  major  design  reviews  such  as  the  PDR  and  COR.  He  must  insure 
that  his  contractor  counterpart  is  a  significant  participant  in  the  presen¬ 
tation  of  program  data.  The  increased  attention  and  emphasis  on  evaluation 
during  early  design  phases  have  led  to  the  frequent  use  of  mockups  to 
assist  in  design  evaluations.  Early  development  of  mockups  is  required  in 
the  full  scale  development  phase  and  helps  to  serve  as  a  design  configura¬ 
tion  aid.  The  Air  Force  HE  manager  may  also  wish  to  attend  certain  test 
and  evaluation  events  which  are  significant  to  the  man-machine  interface. 

He  may  initiate  design  review  unsatisfactory  reports  (i.e.,  deficiency 
reports).  He  may  participate  in  the  initiation  (by  other  Air  Force 
managers)  of  ECP's  when  required. 

Frequently,  the  system  design  will  progress  by  means  of  an  evolving 
baseline  configuration.  The  baseline  will  probably  start  as  that  indicated 
in  Section  3.9.8,  Baseline  Monitoring.  In  order  to  insure  that  all 
subsystems  or  elements  of  the  WBS  are  directed  toward  the  same 
configuration,  a  baseline  with  configuration  control  is  maintained.  It  is 
modified  only  with  agreement  of  all  affected  and  the  modifications  are 
published  for  information  and  review  to  those  organizations  that  should  be 
involved.  It  is  part  of  the  Air  Force  HE  manager's  job  to  keep  track  of 
this  baseline  configuration  and  to  insure  that  there  are  no  potential 
existing  HE  problems  associated  with  the  design. 

During  the  period  of  design  reviews  (or  at  any  convenient  time),  while  the 
Air  Fores  HE  manager  is  visiting  the  contractor,  the  contractor's  HE  data 
file  should  be  reviewed.  This  file  should  contain  copies  of 
correspondence,  reports,  analyses,  specifications,  sketches,  drawings, 
checklists,  and  test  data  reflecting  HE  actions  and  decision  rationale. 

This  review  time  can  be  well  spent  to  assess  how  well  the  contractor  is 
doing  his  job. 
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Generally,  during  the  period  of  program  acquisition,  the  Air  Force  ME  man¬ 
ager  is  available  to  answer  contractor  questions,  ,  provide  certain  Air 
data,  and  give  advice.  However,  in  recent  yea^r^'a^ey  program  acquisition 
phases  have  been  completed.  Hardware  has  been 'designed  and  prototypes  con¬ 
structed  for  a  fly-off.  In  this  kind  of  a  competition.  It  Is  extremely 
difficult  for  the  Air  force  HE  manager  to  provide  help  to  one  contractor 
without  being  very  sure  that  the  same  help  or  information  is  provided  to 
the  other  contractor(s) .  In  this  situation,  the  total  efforts  of  the  Air 
Force  HE  manager  must  necessarily  be  much  greater  than  if  there  were  no 
competition. 
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